Microsoft Word - JDL08_HigginsWORD.doc Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 1 Staff and Student Views of AUTonline (BlackBoard) after Three Years ANDREW HIGGINS, JANNEKE KRIEG AUCKLAND UNIVERSITY OF TECHNOLOGY AUCKLAND, NEW ZEALAND INTRODUCTION This paper reports on the findings of two studies conducted by the Auckland University of Technology (AUT) Institutional Research Unit (IRU), in conjunction with the School of Education, into staff and student satisfaction with e-learning provision through the AUTonline (BlackboardTM) learning management system. BACKGROUND In 2002 AUT acquired a learning management system intended for use by the whole university. At that time, AUT operated at least two learning management systems (LMS) in separate parts of the institution. Senior staff at AUT chose to adopt a single LMS for three general reasons: first, to enable students to gain access to the world of electronically developed and formatted documents; second, to demonstrate that AUT could exhibit national leadership in flexible learning; and third, to make an electronic contribution of learning materials to the Global University Alliance (GUA), of which AUT was then a member. More recently, the Joint Information Systems Committee (JISC) of the United Kingdom has reported a number of tangible benefits arising from e-learning. JISC reviewed 37 case studies and finds: The most fundamental point to come out of all of the case studies is that the appropriate use of technology is leading to significant improvements in learning and teaching across the sector and this is translating into improved satisfaction, retention, and achievement. E-learning is facilitating the expansion of the sector without necessitating corresponding increases in the footprint of the physical estate and it is allowing broadly the same numbers of staff to educate a larger and more diverse student body. The kind of high quality, diverse, accessible, expanding higher education system desired by government and funders is no longer possible without e-learning. (JISC, 2008, p. 33) According to Shephard (2001, p. 161): Today’s technology should enable students to incorporate a range of memory-intensive media into their assignments and research reports and this process may be highly beneficial to learning and assessment. Widespread use of this technology, however, is likely to depend on solutions to difficult copyright restrictions, adequate Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 2 training of students and staff as well as access to suitable hardware. Similarly, Lee (2001, p. 153) writes, “Findings indicated that faculty motivation and commitment toward distance teaching were strong in general. However, faculty motivation and commitment were higher in the institutions with well-provided instructional support.” Along with the JISC report, these two studies show that it is expected that improvement in retention, achievement, and satisfaction are likely to result from the use of e-learning strategies. AUSTRALASIAN CONTEXT Smith, Ling, and Hill (2006) examined six Australian universities using multiple modes of delivery to students, including e-learning, distance education, multi- media, and face-to-face teaching. Smith et al. found that the institutions had an overwhelming ongoing commitment to face-to-face teaching: While face-to-face teaching may be supplemented by other resources, especially an increasing use of on-line materials, the core teaching methods remain quite traditional. Universities emphasised the fact that this mode of delivery is expected by students and is economical to use. (p. 76) It might be thought that AUT, as a multi-campus institution, is committed to face-to-face teaching as its predominant mode, which was the case in 2002. Nevertheless, AUT’s documents indicate that meeting learner needs is important and that flexible delivery methods help meet those needs. In addition, AUT believes that using more e-learning strategies will provide opportunities for staff to use more innovative teaching approaches, when they are aligned with appropriate teaching methods in which learners play a more active role in their learning. AUT also believes that its student body has become more diverse, attracting more Maori, Pasifika, and Asian students than it has in the past. The increased diversity of the student population is in accordance with the findings of Smith et al. (2006). The universities in the Australasian study acknowledged the importance of students gaining generic information technology skills for employment purposes. The use of multiple modes of delivery appeared to enhance those skills. The universities also recognised the importance of staff development in changing from traditional face-to-face teaching toward multiple methods of teaching and delivery. The transition appeared to be smoother in universities already engaged in distance education. With regard to staff acceptance of flexible learning, Smith et al. (2006) believe there is no real evidence that senior managers set out to address the concerns held by staff, especially of workload, in a systematic way. The Pro Vice Chancellor for Online Service at one university stated: There’s people at all different levels. If you go to the faculty of Business and Law, you find most of the staff are at an advanced level because they have been doing it for so long. . . . In the faculty of Arts, you find lots of people doing it but also you would find this is the faculty with most resistance in it as well. But my general view is that most of the staff have embarked on it and want to use it. (p. 78) Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 3 AUT’S PLACE IN REGIONAL E-LEARNING DEVELOPMENT A study by Marshall (2006) examines the extent to which AUT has matured, in a systems sense, as an e-learning institution. Marshall’s comparative study reveals that AUT has developed an e-learning system somewhat in advance of other New Zealand universities. The Marshall study places AUT advantageously in the New Zealand context for its maturity in e-learning systems, noting however that more work needs to be done to enhance both pedagogical uses of the technology and means to communicate success to others. Despite this, AUT’s dominant mode of teaching remains face-to-face, mainly because that is what the staff members know how to do and because it is what students aged 19 to 25 years expect. Older students typically are less able to attend campus-based teaching sessions for various personal, professional, and geographic reasons and are more likely to study from home. In this respect, AUT is very similar to other universities in the Australasian region, except those well versed in distance education, where home study is common. However, as the AUT survey demonstrates, online learning is gaining ground quickly among staff and students as a significant adjunct to face-to-face teaching. The Australasian report states that the institutions studied “made some progress toward achieving the outcomes originally expected from the development of multiple modes of delivery and have begun to make other related changes in teaching and administrative practice as a consequence of this experience” (Smith et al., 2006, p. 79). AUT finds itself in a very similar position. Recent work by the JISC, as noted above, indicates that e-learning provides significant benefits to tertiary institutions, although they have to make substantial changes to the ways teaching and learning are conducted. More specifically, the blending of strategies using digital technologies is the more acceptable route for largely on-campus teaching institutions. Additionally, the intro- duction of a learning and content management system impacts on several other institutional digital processes, such as the student management system, library systems, and institutional record- keeping and document storage systems. Institutions engaging in e-learning might be aware that more than teaching and learning will change because a learning and content management system is introduced. As more staff adopt digital teaching strategies, student demand for them rises much more quickly than might initially be anticipated, producing additional workload for staff development units to upskill staff in e-learning activity. UPTAKE OF AUTONLINE Since its initial implementation in early 2003, the use of AUTonline has grown to the point where it is an essential piece of how AUT does its business. With Blackboard as the foundation learning management system, usage has grown from about 30 courses by the end of 2003 to more than 500 courses for the first semester of 2006, and about 650 courses for the second semester of 2006. While the extent to which AUTonline is used varies greatly from course to course, reports show that approximately 8,000 users (staff and students) were active across a normal semester for 2006. For both course and user numbers, this represents between 40 and 50 percent of total AUT business. By 2008 an analysis Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 4 of AUTonline use indicated that 70 percent of all academic staff used the learning management system. It is expected that by 2009 all academic papers will have an AUTonline presence. The Studies AUT’s Institutional Research Unit conducted two studies in 2006 to establish the extent of both staff and student satisfaction with the use of the LMS, in this case Blackboard. This survey addressed, in part, the question of access to educational resources and systems from staff and student perspectives. There were 216 staff responses to the survey or 54 percent of those 400 staff surveyed. AUT’s Institutional Research Unit identified approximately 8,000 students using AUTonline regularly and surveyed a random sample of 1,500 students. A total of 499 (33 percent) of the selected students responded to the survey. Staff and students received questionnaires containing 80 or so questions, some of which were factual identification items. Most were Likert Scale questions (using a scale from one to seven), and each questionnaire contained several options for open-ended text responses. Some of the responses are copied below. As with all similar studies, there were limitations attached to the survey sample. The results of the staff survey are indicative of the sample used (i.e., known users of AUTonline) and therefore contained few extreme views. Had the sample included staff who, for a variety of reasons, did not or would not use the platform, the results may have had a different emphasis. Although the response rate was good (54 percent), this represents only 216 members of the staff who use AUTonline, which is a moderate proportion of the total academic staff of more than 900 members who also had opportunity to participate. The student study included students who used AUTonline for a given time or who used it at least more than the chosen frequency. The views of non-users or those who used it infrequently are not represented. AUT STAFF SURVEY The following figures show the overall satisfaction findings from the two surveys undertaken at AUT in 2006. Figure 1 Staff survey (mean responses) Overall Satisfaction with AUTonline Likert Scale (1 = strongly disagree; 7 = strongly agree) Ease of Use 5.41 Uploading Confidence 5.63 Alterations Confidence 5.63 IT Training Available 5.07 Individual IT Competence 4.97 Time Efficiency 5.12 AUTonline as Teaching and Learning Tool 5.40 Technological Performance 5.32 Expansion Beneficial 5.42 Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 5 The staff at AUT University reported they found all the items to be satisfactory or better, and no items fell into the unsatisfactory range of less than four on the scale (see Figure 1). This survey established that AUTonline has been successfully implemented and operated at AUT University. More than 200 staff responded to a comprehensive survey of their attitudes toward the use of this system. Staff used the system to place course announcements online, put lecture notes online, engage in asynchronous discussions online, as well as communicate with students through email. Staff used the system less for assignment work and least of all for multi-media purposes. Eighty percent of the respondent staff found AUTonline easy to use, with it easy to upload materials and to make alterations when necessary. It is significant that the staff respondents did not believe they were as competent as they would have liked to be in their information technology skills. On average more than 70 percent of the respondents believed that AUTonline facilitated student learning. Sixty-six percent believed it helped students learn how to use technologies. Seventy-two percent believed the system made it easier for them to teach their courses. Seventy-three percent of staff reported using AUTonline was not tedious and 67 percent thought the system was easy to use. Twenty percent experienced some difficulty with server problems and 31 percent believed AUTonline enhanced interaction with students. (Note, it is not a corollary that the remaining staff found AUTonline hindered interaction with students.) Eighty-one percent believed their learning materials were suitable for online teaching. Staff using AUTonline responded to questions about their vision of the future of the system. Eighty-four percent believed AUTonline improved the flexibility of on-campus teaching. Sixty- eight percent reported AUTonline helped the university keep up with modern learning and teaching technologies. Surprisingly, for a non-distance tertiary institution, 64 percent of respondent staff thought AUTonline would enhance distance learning. Access to learning materials for international students received 48 percent support. Thirty-eight percent believed AUTonline could be used to enhance workplace learning. Respondents reported concerns about staff workload and limited staff technology skills, and these, along with concerns about the quality of student learning, might be impediments to the further expansion of AUTonline. Seventy-two percent of staff respondents believed expanding AUTonline would be beneficial to teaching at AUT University. At the conclusion of the survey, respondents were asked for any further comments. This section consisted of two open-ended questions. Many responses were in regard to technical capabilities and obstacles of Blackboard. The other major theme that came through was comment regarding training and support. Some of the responses include: • We need to keep on learning so we can keep improving our online components. • Time to attend course. Prefer one-on- one help/tuition. • Staff could be better supported with more extensive documentation, in addition to training opportunities. • Training course and available documentation tend to focus only on the basic tasks. Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 6 • When I need to know how to do something, I usually need it NOW and accessing help is a barrier. The final question of the survey asked the respondents to provide further comments relating to their experience as a staff user of AUTonline. The following responses demonstrate concerns and rewards of use of AUTonline: • The forum is what I have used most. Student-centred learning should be enabled by this—but the students don’t get it yet. • Once semester is underway I do not have time to do courses or upskill. • Working independently (as online requires) does not suit every student and requires self-motivation. • Encourages students to be proactive and self-responsible. • The biggest issue restricting the development of online resources in my area is the lack of time—staff have to develop material. The university needs to acknowledge that staff require time away from other responsibilities to develop skills/ knowledge in using online resources and then time to develop the actual teaching material that will go online. • The support of the FLA is what got me going—it allowed me to hit the ground running. Without that I would still be putting off the introduction of AUTonline. • AUTonline reduces face-to-face interaction, but it increases probably 50-fold the general interaction with students. I get many more e-mails and phone calls than I ever do in the classroom setting. I have the privilege to come to know my students online in a very real and human way. In the classroom I am performing; online with one-to-one e-mails I am reflecting and communicating at another level. Figure 2 Student survey (mean responses) Overall Satisfaction with AUTonline Likert Scale (1 = strongly disagree; 7 = strongly agree) Ease of Use 5.48 Your Usage Experience 5.42 Your IT Skills 5.61 Training Available 4.61 Time Efficiency 5.04 Assisting Learning Needs 5.44 Technical Performance 5.33 More Courses Online 5.08 AUT STUDENT SURVEY As with the staff survey, all items received positive responses above the 4.0 cutoff (see Figure 2), demonstrating a high general level of satisfaction with flexible learning. Overall, students were very satisfied with AUTonline. The highest satisfaction rating of 5.61 was given to students’ satisfaction that their current IT skills were meeting the requirements of AUTonline. Students also were very satisfied with the ease of use of Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 7 AUTonline (mean of 5.48). The response for using AUTonline as a tool assisting their learning needs was positive (5.44), as was their experience using AUTonline (5.42). Students were least satisfied with the training that they had received for AUTonline; however, with a mean of 4.61 this is still an adequate result. The final section of the questionnaire invited students to add any extra comments relating to AUTonline through two free-text response questions. 1. Are there any aspects of AUTonline that have disadvantaged your learning? • If you get stuck or confused, you can’t talk immediately to someone about that issue. You have to e-mail or discuss and wait for a reply. • Lecturers speak less because they think everything is on AUTonline. • Sometimes the teachers can be a bit unreasonable. Some teachers want assignments done online, but not every student is computer literate. • There needs to be a specific area which pops up and tells you that you have such-and-such to do for online work with a link. • Perhaps if the course did not use slides as much; some computers do not have PowerPoint installed. • Not knowing how to use it well enough. 2. Further comments about using AUTonline. • It would be better if other subjects used AUTonline as well, but I think AUTonline should only be resources, not the main method of learning. • I’m an on-campus student, spend a lot of time travelling to school and hanging around the campus to wait for lectures. Increasing online learning would be more efficient time-wise. • Having resources more available to students and having the option to communicate with other classmates and lecturers made learning less stressful. • I live one hour away from campus, so to travel in for a two-hour class can be expensive and time-consuming. Great job! • I feel it’s a good way to get students more involved within AUT and it gives students the opportunity to use those resources that would be helpful for our learning, e.g., lecturer notes, discussion boards of important questions, and answers that allow a whole interaction with the lecturer. • Overall AUTonline is good, especially when used in combination with face-to-face classes. However, down- loading lecture notes and readings is time-consuming and printing them out is costly (paper and ink). It is easier to study from printed notes and easier to add extra notes during class. CONCLUSIONS ON AUTONLINE Staff uptake of AUTonline since its comprehensive introduction in 2003 has been both substantial in numbers and significant for teaching strategies. Strengths have emerged in users’ confidence and the stability of the learning management system. There is considerable support among staff for an expansion of AUTonline. Weaknesses include a perceived inadequacy of staff training for information technologies in general. Additionally, there is anecdotal evidence that issues of benefit and disadvantage in terms of staff workload warrant further exploration, because staff reported they saw workload as the main factor limiting uptake of AUTonline. Teaching changes include more flexibility in the time and place of access to Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 8 learning, time savings in the use of the Gradebook, and online marking. Links to the library’s e-Reserve system are making high-quality resources available to students electronically. A teaching forum entitled “If I put my lecture notes online, will the students come to lectures?” demonstrated that the issue of pedagogical strategy is now more openly discussed than before. Student views about the use of AUTonline are very positive overall. Strengths include ease of use, enhanced IT skills, and assistance with learning needs. Students believe that AUTonline should be used in more courses. Weaknesses include a perceived lack of training. Anecdotal evidence suggests that different groups of students use the learning management system in different ways. Students in some categories report that lack of bandwidth at home (or the ability to pay for home computers and connectivity) is a restricting factor for home-based access, while others acquire more connectivity and rely on AUTonline at home as well as on campus. More widespread and affordable access to greater bandwidth before 2010 will see a growth among students in more home use of the learning management system. Additionally, students reported that a lack of desire to use AUTonline could be a limiting factor in its further uptake. Students also reported that improved AUTonline use would be enhanced by increased guidance from teachers in its use, and that such guidance would have more impact than increased bandwidth. For students, access to learning infor- mation outside of lecture times helps achieve learning outcomes, and learning is accessible to those who cannot always attend on-campus sessions because of work or family commitments. A number of specific recommendations arise from this study. The first recommendation builds on the general acceptance and satisfaction among both staff and students. It is for the university to encourage more use of the learning management system, because it is helping AUT achieve its goals. The second recommendation seeks to enhance existing information technology training practices. The rapid introduction of new or updated software now requires an online, just-in-time training programme based on the latest information. One of the main purposes of using digital teaching strategies is to enhance student learning. In some ways using the technologies leads to changed pedagogy. Exploiting and using these changed understandings of teaching and learning is proving to be both challenging and exhilarating for staff and students alike. A recommendation found in Marshall’s (2006) report and in the 2006 audit of AUT’s learning and teaching activities (conducted by the New Zealand Academic Audit Unit) proposes that AUT find efficient and effective ways of informing and engaging staff about practices emerging among their colleagues. In general, it appears staff development seminar-based strategies, although effective for the small numbers attending and favoured by half of the respondent staff, do not reach the bulk of academic staff. Last, insofar as staff are concerned, the changing character of their work brought about by the technologies requires re-evaluation of teaching. It would be disconcerting to find that e-learning simply adds to the current workload rather than transforming it. Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 9 These issues are being addressed in 2007 and 2008 in two ways. First, the university is conducting research into a comparison of face-to-face and online staff development strategies to compare learning retention and outcomes. Not surprisingly, preliminary results show that staff having the greatest familiarity with information and communications technologies enjoy using them. Second, funds have been allocated for the acquisition of suitable online staff training in the use of generic IT software, such as word processing, Powerpoint, and spreadsheeting. Recommendations from the study that affect students concern the decision to continue using the BlackBoard learning management system. • All university papers will have an online presence in 2009. Pressure exists from some parts of the university to adopt more than one LMS. The survey recommends that it would be in the interests of staff and students to have one common system for login, authentication, look and feel, and ease of training. Although the products AdORE and Moodle are used in localised limited ways, there is no identified need to change LMSs for financial, pedagogical, or efficiency reasons. • Second, to help students best use the e-learning environment, an improved training programme for information technology in general and not just for the LMS needs to be created with guidance from teachers. Such a programme would enhance the existing one. • Third, the diversity of AUT’s student population, which includes Maori, Pasifika, Asian, and European students, should account for the already known social, cultural, financial, and other variations among and between these groups. • Fourth, although most AUT students have computers and all students have access to campus-based computer laboratories, there is an identified need to encourage students to have up-to-date machines capable of handling new software releases for which the university has licenses. Assistance for students to acquire personal computers, preferably laptops, is being negotiated with vendors and leasing companies. The university is providing scholarships with laptops for students in some graduate programmes from 2008. • Last, although it is out of AUT’s direct control, pressure should be placed on the national network providers to ensure more bandwidth is made available in AUT’s student catchment and accommodation areas. IMPLICATIONS FOR THE FUTURE OF E-LEARNING AT AUT The outcomes from the AUTonline reports have implications for AUT’s staff, students, and systems. The power of the Blackboard learning management system to give students access to digitised resources, both of their own creation and from elsewhere, is changing the nature of learning, allowing it to become more lifelong and accessible in character. In particular, AUT will be mandating an AUTonline presence for all of its papers in 2009. The use of a content management system allows for the automated uploading of significant information about each paper and the courses offered, so that students can see the intended outcomes, assessment schedules, and other related materials. Staff members are already using the content management system to upload Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 10 course content. Recent developments in e-portfolio software have the potential to change the relationships between students, staff, and their learning careers. The capacity of e-learning systems to change how staff approach their teaching requires a new evaluation of staff development strategies. New pedagogies and enhanced communication between students and staff, as well as within those groups, indicate that staff members learn extensively from one another directly and with less mediation through formal staff development sessions. On the other hand, staff training to achieve the most from emerging software should be delivered “on the job” and “just in time,” in accordance with practice common in industry or businesses commensurate with AUT’s size and complexity. Changes emerging at the university system level as a result of e-learning are more complex. Staff performance and promotion systems will be changed through the use of e-portfolios. Library systems will be better able to store, monitor, and protect the use of digital resources, both as university assets and to ensure compliance with legal require- ments by using various forms of content management systems. The information technologies underpinning AUT’s admin- istrative systems need to be as modern as possible and very robust to ensure that e-learning systems function well. Finally, research into the ongoing impact of these challenges should monitor institutional health for the benefit of all those involved in the life of AUT as it is changed by e-learning strategies. REFERENCES Joint Information Systems Committee (2008). Tangible benefits of e-learning: Does invest- ment yield interest? Retrieved March 6, 2008, from http://www.jisc.ac.uk/ publications/ bptangiblebenefitsv1.aspx Lee, J. (2001). Instructional support for distance education and faculty motivation, commitment, and satisfaction. British Journal of Educational Technology, 32(2), 153–160. Marshall, S. (2006). New Zealand tertiary institution e-learning capability: Informing and guiding e-learning architectural change and development. Wellington, New Zealand: Ministry of Education. New Zealand Academic Audit Unit (2006). Auckland University of Technology Audit Report, Cycle 3. Wellington, New Zealand: New Zealand Academic Audit Unit. Shephard, K. A. (2001). Submission of student assignments on compact discs: Exploring the use of audio, images, and video in assessment and learning in higher education. British Journal of Educational Technology, 32(1), 161–170. Smith, A., Ling, P., & Hill, D. (2006). The adoption of multiple modes of delivery in universities. Journal of University Teaching and Learning Practice, 3(2), 67–81. Andrew Higgins is the Director of E-Learning at AUT University. He has published various articles discussing distance and e-learning. Janneke Krieg is a Research and Data Analyst in the Institutional Research Unit at AUT University. Journal of Distance Learning, Vol 12, No 1, 2008 © Distance Education Association of New Zealand 11