Microsoft Word - holt.doc Australasian Journal of Educational Technology 2007, 23(1), 110-131 From policy to practice: One university’s experience of implementing strategic change through wholly online teaching and learning Dale Holt and Di Challis Deakin University One institution’s attempts to implement an ambitious wholly online learning policy is examined in the Australian higher education setting. The conditions that led to a diversity of models of wholly online unit development are considered, along with teachers’ design intents in establishing their online teaching and learning environments. The emphasis of the analysis is on those wholly online units created or redeveloped from existing offerings which held out the greatest possibilities for what we have categorised as ‘transformatory’ learning and teaching as related to the development of highly valued lifelong learning capacities in students. Organisational learning from the offering of the first major suite of wholly online units is outlined and impacts on policy reformulation described. The case study concludes with a consideration of more general lessons learnt from policy driven initiatives directed at transforming teaching and learning in higher education. Introduction In 2003, Deakin University developed an ambitious plan to move more systematically and substantially online for teaching, learning, academic support and administration. One notable policy objective was to ensure that every undergraduate student enrolled at Deakin from 2004 completed at least one unit wholly online. An indication of the seriousness of this requirement was that exemptions to this requirement could be granted by senior academic management only. In the context in which Deakin University chose to adopt its proprietary learning management system (LMS), repackaged as Deakin Studies Online (DSO), and to develop a policy of online learning, “learning ICT and learning through ICT was such a natural combination that eLearning became a star in the policy discourse of the years 2000 and 2001” (European Holt and Challis 111 ODL Liaison Committee, 2004). At that time a survey of online education in Australia conducted by DEST (2002) found that there were 207 fully online courses, mainly at postgraduate level, offered by 23 of the 40 Australian universities who provided data. Seven universities used the Internet for 99 or 100 per cent of their units, with the lowest usage being 9%. The continued growth of Internet technologies within the education sector has led many institutions, such as ours, to adopt a learning management system. While, as the European ODL Liaison Committee paper points out, the projected 100% annual growth is approximately 30% in reality, in the past ten years learning management systems [LMS] have had such widespread and rapid adoption they have become almost ubiquitous in many parts of the world with recent estimates suggesting that in many countries about three-quarters of institutions have an LMS (Coates, 2005). However, the investment in such a system does not, of itself, assure meaningful eLearning. Referring to consolidated studies in the higher education sector, the European ODL Liaison Committee (2004) concluded that “leaders of universities are badly equipped and supported to implement and mainstream changes needed for successful introduction of meaningful eLearning in their institutions”. As Bonk (2004, p3) argued: Unfortunately, most online learning technologies are designed for the management of learners online, not for online learners to manage their own learning. Too often online courseware provides access to student records or tests, but not rich, interacting learning experiences. Consequently, too often there is no learning in e-learning. Our interest was in how a mandated movement to online learning through an LMS would play out in practice and whether the experience would be ‘successful’, ‘meaningful’ and result in ‘rich, interacting learning experiences’ - learning that we characterise below as ‘transformatory’. In their commissioned study (DEST, 2003), the University of Southern Queensland [USQ] reported on their approach to offering courses totally online. Although they acknowledge the limitations of their study (one faculty with eight courses at postgraduate level) they consider their experiences have relevance to all institutions teaching online. Our study of the teaching and learning experience in all wholly online units across one university is instructive in two important respects. First, in common with the USQ/DEST study, it sheds light on how teachers and learners experienced this novel teaching and learning environment with implications for the design and conduct of such environments which make substantial use of digital media and online technologies. More broadly based, it reveals more fully the difference of the wholly online experience and, in contrast to the USQ/DEST study, it concerns undergraduate students. Second, it provides powerful insights into how any form of policy innovation can be interpreted and enacted in multiple ways which may not 112 Australasian Journal of Educational Technology, 2007, 23(1) necessarily be in accordance with the letter of the original policy intervention and which, in themselves, can influence the (re)interpretation and refinement of such policy. Research methodology This research continues the authors’ involvement in the use of case study methodology to illuminate participants’ experiences of significant teaching and learning developments and initiatives in Australian higher education (see Holt, 1993; Holt & Thompson[1], 1995; Thompson & Holt, 1996; Holt, Rice, Segrave & Thompson, 1997; Holt & Thompson, 1998; Holt, Mackay & Smith, 2004; Challis, Holt & Rice, 2005; Smith, Mackay, Challis & Holt, 2006). The case study methodology is situated within the interpretive paradigm of educational research and practice (see Carr & Kemmis, 1986, Chapter 3). The experiences of teachers and their learners of wholly online environments at the undergraduate level is the focus of this research. Both authors have played an active role in the developments of wholly online units and in supporting an institutional initiative, the Online Teaching and Learning Fellowship Program 2003-2004, which fostered a community of interest around innovative uses of educational technologies (see Segrave, Holt & Farmer, 2005). Our role in this research is primarily as the interpreters and illuminators of the meanings that teaching staff ascribed to the wholly online policy imperative and how they went about enacting their own quite varied representations of wholly online environments in their own local situations and often in constraining circumstances. We have developed frameworks for understanding and explaining how teachers ascribed multiple meanings and pursued a diversity of actions in their enactment of the policy in the realities of their practice settings. The discrepancies or incongruities between the one policy but multiple enactments are highlighted. We do, though, acknowledge our role as agents for encouraging ongoing discussion and change in relation to the development of this type of environment. Hence, our ‘interviews’ with staff adopted a conversational, negotiated, forward looking and improvement oriented style. Walker (1985, p.118) sees such a style of open, semi- structured interviewing as involving the framing of interviews ‘in ways that at least provide the opportunity for reflection and processing within the interview itself. For example, pausing at intervals and asking the interviewee to recapitulate and to summarise, or even offering summaries and asking for an assessment of your understanding; explaining initially what your intention is and asking for critical responses; leaving tapes with the interviewee to allow him or her to listen back to what was said and to comment on it; corresponding with people after an interview.’ We concur with such a style of interviewing (see Thompson, 1996). Holt and Challis 113 The research builds on pilot surveying undertaken in one of the first wholly online units offered within the University and reported by Armatas, Holt and Rice (2004). A representative selection of wholly online units by eleven key disciplines and professional fields was chosen as the focus of the research in 2005 (see Table 1). Seventeen Faculty based academic teaching staff members were involved in 15 interviews lasting on average one hour. These discussions were conducted after staff members had completed at least one round of teaching of their respective units. The interviews were recorded using a portable digital recorder, transcribed and analysed following confirmation that each transcription provided an accurate impression of the staff member’s viewpoints. Although conversational in style, staff were informed prior to the interview that the following areas would be discussed and, while the sequence varied, each of the following areas was covered. 1. Rationale for unit being a wholly online unit. Changed views on rationale after teaching the unit. 2. Knowledge of students taking unit. How knowledge of students influenced approach in a wholly online unit. 3. Key learning outcomes of the unit. How knowledge of wholly online learning impacted learning outcomes and if/how desired learning outcomes impact on how unit was developed online. 4. Process of developing wholly online unit, including role. Key design considerations in selecting, organising and structuring the subject. 5. Main drivers in organisation of assessment. Design and implementation of assessment drivers to engage students and enable quality learning outcomes. 6. Experiences of online environment in relation to discussion over the semester. Satisfaction and comparison to face to face. 7. On a scale of 0-5 with 0 worthless and 5 excellent, teacher satisfaction with online unit. Teacher perception of student satisfaction. 8. Enhancement of wholly online teaching and learning experience in future. 9. Any other comments. To gauge student perceptions, an online survey (Experiences of Learning Online Survey) was developed from previous surveys used by the authors (see, especially, Armatas, Holt & Rice, 2004; Challis, 2005) and elsewhere (see, eg, the CSTL Item Bank and the ETL Project questionnaires) and was refined by feedback from several of the wholly online teaching staff involved in the interviews. Seven-point scale items invited students to indicate the degree to which the item attribute was important and then the degree to which they were satisfied with it. On the basis of these questions, a weighted average satisfaction index was calculated, overall and for each unit individually. Of the 8 units surveyed, 389 responses were received from an estimated total student population across all units of 2800 (a 114 Australasian Journal of Educational Technology, 2007, 23(1) response rate of 14%). We draw tentative conclusions from this data pool about the wholly online student learning experience relating to the group of units surveyed, especially those concerning what we have designated as ‘transformatory’ learning. We acknowledge the difficulties of achieving higher response rates (even with reminder email) given the survey must be offered after the University’s mandatory institutional student evaluation surveying cycle, it being online and it being opened up when students may have completed their engagement with their online learning environment where the survey was placed. The survey has been re-administered with wholly online units in 2006. Detailed analysis and reporting on the expanded data pool will be presented elsewhere, and falls outside the focus of this paper. Defining wholly online environments During the formulation of Deakin University’s online technologies policy in 2003, the original definition of wholly online units was as follows: A3. An Online Unit (wholly online) These units will have all teaching resources and undertake all teaching online including: • All content (either commercial print-based textbooks or commercial e- texts may be used as supplementary material) • All communication and interaction with students • Assignment submission and feedback (examinations will move online when the University is administratively ready) • Each unit will have at least ONE session of interactive communication (synchronous, asynchronous, or both) between teacher and students online at least weekly or as established at the beginning of the course. Such interactive sessions will have an assessable component where appropriate. To ensure access for all students until bandwidth issues in Australia are addressed, additional resources such as video and audio will be provided on CD-ROMs for off-campus students where appropriate. (This will be reviewed annually.) (Deakin Online Technologies in Courses and Units – Operational Policy, Approved by Academic Board on 25 July 2003, Appendix A: Deakin Categories of Online Activity). This is a quite detailed and prescriptive policy definition the core of which requires that no wholly online unit can or should offer face to face teaching. Moreover, the definition strongly suggests that the wholly online environment was to be distinctive in terms of its experiences for teachers and the different cohorts of students who would learn in these environments. It is not contended that offering courses and units wholly Holt and Challis 115 online is a novel development in higher education nationally or internationally. What is novel is the requirement that all students in undergraduate studies must undertake one unit wholly online, whether their overall mode of enrolment c h o i c e is internal/on campus or external/off campus. They are required to partake in a learning environment which withdraws the interpersonal communication expected of on campus study and which provides limitations on flexibility and independence of study expected of the off campus experience. The distinctiveness of the wholly online environment was to lie in a design aspiration to develop further lifelong learning capacities and to do so in ways which did not replicate either a traditional online supported face to face teaching/learning environment for on campus, predominantly school leaver students, or a ‘traditional’ self paced, independent learning experience for predominantly mature aged, off campus students. In relation to lifelong learning capacities, senior management and policy makers in the University argued during this period that a wholly online unit should allow students to: • acquire and practice a range of technical skills needed to work effectively in online environments; • develop an understanding of issues and learn to act in an ethical and responsible manner in virtual environments; • develop skills in online communication; • develop a capacity for online teamwork, collaboration and negotiation; • learn to assess and evaluate the quality of online information; and • develop the organisational and personal management skills necessary to sustain motivation and study effectively and successfully without regular face-to-face contact with teachers and other students. At the University’s Online Teaching and Learning Fellowship opening session in March 2004, the then Pro-Vice-Chancellor (Online Services) exhorted the Fellows to take the opportunity to be innovative in their online teaching and learning projects. Many of these Fellows were developing wholly online units and this exhortation was interpreted as providing licence for the development of a diversity of models of wholly online design and operation. This senior management signal could reasonably be seen as encouraging developments somewhat inconsistent with the prescriptive definition ascribed to going and being wholly online, promulgated in policy a year earlier. Intended or not, this has in turn led to a reconsideration of the policy definition of wholly online in 2006, which is considered further below. 116 Australasian Journal of Educational Technology, 2007, 23(1) Profiling the wholly online units Issues for discussion with academic teaching staff were strongly influenced by the work of Biggs (2003) on constructive alignments amongst key dimensions or areas of the wholly online teaching and learning environment. An appropriate alignment of these areas is seen to afford the best opportunities for quality student learning. The interviews, therefore considered the unit’s rationale and curriculum, pedagogies, media and technology mixes, online discussion design and moderation, and assessment approaches both individually and through exploration of their connections. All of this was considered in the unique context of wholly online unit development and operation. Moreover, discussions explored staff member’s own evaluation approaches to their unit and the unit improvements they had in mind for future offerings (these intended improvements are set in the context of student experiences in the next section). Table 1 summarises unit areas of alignments. Table 1: Profiling wholly online units Discipline Field Rationale Curric- ulum Pedagog- ical basis Media Online Interaction Assess- ment First year units Science Skills [Interview 1] Develop generic scientific and learning skills Sequential around scientific process Didactic and enquiry based Text, multimedia presentations Bulletin boards and chat tutes Information treasure hunt, report, online tests and exam Health Sciences [Interview 2] Information and IT literacy in discipline context Sequential around scientific process Challenge/ task based Video presen- tations and text on CD Bulletin board support and for student feedback on unit Online tests, report and exam Second year units Enginee- ring [Interview 3] [Interview 4] Develop lifelong learning and generic skills required by profession Sequential Modular for key areas of engineer- ing manag- ement Didactic Animations, video docs, radio interviews and text on CD. Audio topic overviews online General Bulletin board support 2 applied assignments and exam Information Systems [Interview 5a#] [Interview 5b#] Promote unit and School, lends itself to online with no lab work Sequential Key areas of networking Enquiry based Cartoon teacher Avatar, animations and text all online General Bulletin Board support Online tests, case study report and exam Holt and Challis 117 Marketing [Interview 6a#] [Interview 6b#] [Interv iew 7] Action- oriented discipline Technology for profess- ional learning Sequential Topic based Case based and reflective learning Video presen- tations and cases, text on CD, streamed support for exam General Bulletin Board support and small groups for reflective work and mentoring Online mastery tests, group reflective journal (shared with other groups) and exam Third year units Inform- ation Technology [Interview 8] Technical skills and virtual team skills for employment Sequential Issues based First topic orientation to online learning Enquiry based, discursive All text online only General Bulletin Board support and online groups Online group participation, reports and exam Education [Interview 9] Learning online for all secondary teacher education Thematic Valuing numeracy at work, in the profess- ion and in teaching Video pres- entations and interviews, readings on CD Online tute groups Portfolio, group assign- ment (including peer assess- ment) Arts Sociol- ogy [Interview 10] [Interview 11] Social impacts of Internet taught on the net and critical evaluation of web information Skills in navigating virtual spaces Modular discipline areas Experi- ential Video doc on CD, and text and audio online General Bulletin Board tutes and online tutes Variety of assign- ments (plagiarism and online tests, net use report, case study bibliography and report) Arts Media and Comm- unication [Interview 12] Develop lifelong learning skills as part of program exit strategy Core unit but not profess- ional requirement Sequential by topic Didactic and case- based Audio lectures to slides, law oration, video current affairs, case cartoons, text on CD, and lecture notes, online tests and podcasts online Podcasting, blogging and e-tutes Bulletin boards for topics, assign- ments, exam and online tute groups Online tests (no marks), case assign- ments and exam 118 Australasian Journal of Educational Technology, 2007, 23(1) Law [Interview 13] [Interview 14] Develop lifelong learning skills as part of program exit strategy Core unit but not profess- ional requirement Sequential by topic Didactic and case- based Audio lectures to slides, law oration, video current affairs, case cartoons, text on CD, and lecture notes, online tests and podcasts online Podcasting, blogging and e-tutes Bulletin boards for topics, assign- ments, exam and online tute groups Online tests (no marks), case assign- ments and exam Sixth year units Architec- ture and Building [Interview 15] Profess- ional practice unit on entry to work Sequential by topic Authentic learning approaches Multi- disciplinary Lecture material online and multimedia case on CD/online Multiple concurrent online debates Work in groups, submit individual reflective report #Note: Where ‘a’ and ‘b’ are indicated, this means two staff were interviewed together. The unit profiles immediately suggest key areas of diversity in the interpretation and enactment of policy by the teachers involved. Units differed in relation to: • location within undergraduate program; • the explicit emphasis given to developing lifelong learning capacities as a key learning outcome; • the nature of the structuring and organisation of the curriculum; • the student centred nature of the learning activities; • the extent to which online discussion was provided and engaged with by students; and • the extent to which formal assessment was tied to students’ interactions with digital and online resources, and their peers. These differences can be understood in relation to several factors: • the year level at which the wholly online unit was offered; • the previous experiences that staff had had in teaching online and developing digital media; • the degree to which the unit was created or created anew for wholly online delivery; • the key curricular and pedagogical concerns of the discipline or field; • how adventurous staff were prepared to be in locating formal assessment strongly within online student activity; • how literally staff took the policy requirement to be wholly online; Holt and Challis 119 • the institutional and local opportunities or constraints experienced in the development of their units which meant for better or worse innovation being either inhibited or enabled (see next section); and • the extent to which staff volunteered to offer their unit wholly online as opposed to being asked to do so by their management. What can be concluded at this point is that teacher beliefs, values and practices in relation to educational concerns and the technologies provide only a partial view of why wholly online units turned out the way they did, and, in many cases, turned out quite differently from one other. Institutional policy does not translate into uniform, standard responses because of varied political and cultural conditions discerned in local contexts in different faculties and schools, and on different campuses. While the University of Southern Queensland’s researchers found that the “online teachers [in the one Faculty, Education] were in ‘change mode’; they were not trying to re-interpret teaching and learning around traditional structures, principles and practices” (DEST, 2003, p.xvi), our study revealed diversity. Diversity of development models Four different models of wholly online unit development have been discerned in analysing the staff interviews. These models have been constructed on the basis of the newness of conceiving each area of alignment, and the newness of all areas as aligned, integrated and embodied in each unit’s wholly online teaching and learning environment (see Table 2). The four development models are: 1. Existing unit Deployment model: no change to subject/curriculum, pedagogies, online environment and assessment. Value enhancements to learning resources in digital form. 2. Existing unit Translation model: no change to subject/curriculum and pedagogies. Enhancements to learning resources in digital form, online support and assessment strategies. 3. Existing unit Redevelopment model: changes to pre-existing unit subject/curriculum, pedagogies, learning resources, online support and assessment. 4. New unit development model: new unit created for wholly online (no separate rationales for the unit on the one hand, and it going wholly online on the other) – new in all respects from subject/curriculum, pedagogies, learning resources, online support to assessment. Aimed at making the online and digital a key environment for active participation and study. 120 Australasian Journal of Educational Technology, 2007, 23(1) The extent to which staff could and wanted to undertake significant new developments was shaped by their strength of existing commitment to the unit as previously offered in other forms (where applicable), the support and opportunities provided locally for new developments, and staff members’ own sense of acceptable risk taking in going wholly online. Clearly, those units which were created from scratch with wholly online centrally in mind provided the strongest opportunities to think and design anew for the affordances of digital and online learning. Table 2: Models for developing and operating wholly online units (A) and across an innovation continuum (B) A Deployment model Translation model Redevelopment model New develop- ment model Units B Low innovation Medium innovation High innovation Very high innovation: Transformative Science skills Yes (a) Yes (b) Health Sciences Yes (a) Yes (b) Engineering Yes (a) Yes (b) Information Systems Yes (a) Yes (b) Marketing Yes (a) Yes (b) Information Technology Yes (a) Yes (b) Education Yes (a) Yes (b) Arts Sociology Yes (a) Yes (b) Arts Media and Communication Yes (a) Yes (b) Law Yes (a) Yes (b) Architecture and Building Yes (a) Yes (b) Southwell, Gannaway, Orrell, Chalmers and Abraham (2005, p.2) define innovation as ‘an idea, product, process or service that adds value, is useful or transforms current practice in the context to which it is applied’. In Table 2, an innovation continuum is introduced showing the match between field, development model and level of innovation. It is argued that the wholly online teaching and learning environment, as embodied in University policy, is an innovative context for potential unit development. Actual unit development for wholly online environments exhibits different Holt and Challis 121 levels of innovation within the overall policy context. It is not argued that any particular area of alignment for any of the units is totally new in higher education. Some forms of curricular and pedagogical development might seem more progressive (for example, constructivist or student centred, or student active) than others. Some uses of pedagogy in a particular setting might be innovative as judged by the norms of that disciplinary context as, for example, where problem based learning has been adopted in a field not normally characterised by the use of such pedagogy. Our focus though is on innovation as examined through the alignments amongst curriculum, pedagogy and assessment in relation to perceived affordances of the digital media and online technologies. With but one exception, Information Technology, the model for development and the perceived level of innovation correlates and in this instance the initial development had been with close attention to exploitation and integration of digital media. We have characterised the very high innovation development model as transformative in potential; that is, capable of realising new forms of teaching and learning value through new forms of engagement with digital media and online learning communities. It is not suggested that all that is highly innovative is necessarily good, or that those units characterised by low innovation were not valuable learning experiences for students. Moreover, transformative design intent, as will be explored later, is not necessarily fully realised, at least initially, for a number of factors, including support and infrastructure limitations, and the need for innovation environments to be incrementally developed over time. The mere presence of learning environments saturated with digital media and online features does not represent an inherently powerful teaching and learning environment. Nevertheless, innovative design can be inferred from the definition of wholly online units, was strongly encouraged by senior management, and was offered as a real possibility of enactment through the very need to develop and operate effectively in a teaching and learning environment totally digitally based. Models of transformative design As previously indicated, the wholly online unit policy was promulgated with the intent of establishing a distinctive teaching and learning environment for all student cohorts, to further develop lifelong learning capacities. Candy, Crebert and O’Leary (2004, pp.43-4) profiled effective lifelong learners in their investigations into developing lifelong learners through undergraduate education as having inquiring minds, helicopter vision, a sense of personal agency, a repertoire of learning skills, and being information literate. In relation to wholly online units, the most important capacities or generic attributes to be developed further related to personal management, information technology and information literacy, and 122 Australasian Journal of Educational Technology, 2007, 23(1) communication and collaboration skills. Consistent with Candy et al’s conception, information literacy as grounded in disciplinary concerns featured strongly in the highly innovative wholly online unit designs. Clearly those units created or developed afresh for wholly online delivery meet our definition as being the most innovative in nature. They were - at least in design intent - those with the greatest potential to create the conditions for transformatory learning and teaching experiences. To be ‘transformatory’ the experience should lead students to see the discipline and the world of the professional field with its underlying know how in quite different, more compelling, more enlightened and more relevant ways to their everyday world. Moreover, their designs provided the greatest possibilities for developing in novel ways key lifelong learning capacities. The nexus between innovative and transformatory design intent as particularly related to the development of discipline specific digital and information (including technology) literacies, in its many dimensions as defined by Candy et al (1994), can be seen in the views of those interviewed in the six units we have characterised as transformatory (see Table 2). Significantly, the thrust of the argument was the same, irrespective of discipline. From the Sciences: I think that the internet offers a lot of applied knowledge activities and learning in context of future careers and industry, and there’s a lot of really good stuff accessible online that people do use already… The learning objective was actually to get the students to understand that learning science is not about filling a filing cabinet with facts, but is learning how to look at facts in a conceptual framework in the way scientists look at evidence and use evidence to support theory and that sort of thing (Interview1). .…the unit covered issues to do with health information and data. It seemed, in terms of matching up the modality and the content, it was an obvious marriage that could work to encourage students into this online space with a pretty good rationale that they’re going to be using this space in their professional lives down the track. And so it was not an artificial task to make them go online to look up information and interpret that because that’s how they do it in the future. Now, they seem to have swallowed that rationale fairly well (Interview2). From the Arts: One, we wanted students to step back and consider the social world, even the electronic world, in a less technologically determinist fashion and that was the hardest thing to get across. Many students just take the material as technologically determinist. … Two, the notion that a whole heap of crimes and criminal activity that exists on the net now is somehow new when clearly it is not new. It is a continuation of forms of criminality that are now using different technologies to support themselves ... The third thing that I Holt and Challis 123 really wanted them to consider was just because it’s on the net doesn’t actually make it real, truthful or in any way useful…. so they have to start thinking about it somewhat more reflectively and critically (Interview10). … but given that they’re (advertisements in the print media) of declining significance in contemporary popular culture, it seemed to me only right and proper that we should use an electronic environment because much of advertising - like much of contemporary popular culture - exists in an electronic environment such as online would provide (Interview12). Student responses to questions on the survey relating to the development of generic student attributes of the kind delineated above as ‘transformatory’ can be found in Table 3. Of the 8 units surveyed in 2005, 4 have been categorised as ‘transformatory’ in design and intent. A further 2 were not surveyed at the lecturer’s request. Table 3 shows responses overall and then in bold italics below for the 4 identified ‘transformatory’ units. Table 3: Student responses to ‘transformatory’ generic student attribute questions in survey Question ‘Transformatory’ generic student attribute Importance rating Satisfaction rating Q.7 Being able to access online/digital learning resources readily Information literacy 6.1 6.1 4.9 4.9 Q.8 Being able to learn without regular face to face contact Personal management 4.9 5.0 4.1. 4.2 Q.9 Organising and being responsible for your own learning Personal management 5.9 5.9 4.9 5.0 Q.19 Interacting online with teaching staff Written communication 5.7 5.6 4.4 4.6 Q.20 Interacting online with other students Written communication, teamwork and collaboration 4.9 5.3 4.4 4.0 Q.32 Having the opportunity to develop/ practice online technical skills Information technology literacy 5.3 5.2 4.6 4.9 Q.33 Learning to judge the quality of online information Information literacy 5.5 5.2 4.7 4.8 Q.34 Having the ability to communicate knowledge and deal effectively online Communication, teamwork and collaboration 5.3 5.3 4.5 4.9 On a 1-7 scale, students in all 8 units surveyed gave 5+ ratings for importance to all but two of the questions we identified as especially relevant to this area and these were very close (4.9). However, only one item (being able to access digital/online learning resources readily) scored 124 Australasian Journal of Educational Technology, 2007, 23(1) a 6 (6.1). In terms of satisfaction, the response was more uniform, varying from 4.1-4.9. It is readily apparent that the aggregated responses to the four identified ‘transformatory’ units did not sit outside the norm. However, in four instances, and in three different units, student satisfaction was markedly low, ranging from 3.0 to 3.8. Significantly, no unit for any question recorded a student satisfaction level of 6, with 5.5 the highest for the same unit for three questions and this was for students in their sixth year of study. The 155 additional comments revealed that students’ concerns were mainly with practical rather than pedagogical issues and the realities of the learning experience that were shaped more by personal interaction (its frequency, usefulness and tone) than the technologies. “Keep up the level of enthusiasm and patience by staff members, this makes all the difference” wrote one student. Another told us, “I believe there is significant merit to studying this subject and likewise in conducting wholly online subjects in general, but feel better understandings are needed by those in charge for them to operate effectively”. As indicated by the following student quotation from the survey, the need for wholly online units to provide novel learning opportunities was not missed, but nor was it seen to be experienced in all cases: I didn't see much difference (apart from the student collaboration element) between a wholly online unit and an off campus unit. There was a very large amount of reading to do, and I don't believe that this unit took full advantage of the features and advantages offered by the medium it uses (i.e. the Internet). Certainly there were individual comments that indicated students had recognised some of the learning goals we had associated with transformatory learning. Feedback such as “It has encouraged me to look to myself for more answers rather than the answer being given to me & it has taught me to be more organised and independent” typified this sort of response. But, even in the same unit, the essential difference was the student and their preferred learning style. Comments that praised the opportunity to work online and its value for learning were counter- balanced by those that bemoaned the loss of face to face contact. Statements such as “The interaction with other students, without sitting face to face was great” sat beside comments like “I’ve realised that I strongly prefer learning ‘in person’”. Design aspirations apart, student feedback suggests further work is needed in making congruent teacher design intent and the realities of students’ learning relate to the lifelong learning capacities we have identified as part of ‘transformatory’ learning. Holt and Challis 125 Changing understanding of wholly online as an instance of changing understandings of strategic change in higher education Through the initial period of developing and teaching wholly online units, we have witnessed a diversity of models evolve in response to what at the time seemed like a prescriptive policy imperative laying out both requirement and a detailed form in which such offerings were to be made. Ironically, some of the most innovative unit developments deviated the most from policy prescription. We see this as a key lesson in stimulating change agendas in university teaching and learning. One needs to differentiate between the spirit of policy intent – in this case to develop distinctive teaching/learning environments enabling of the further development of lifelong learning capacities – and prescriptive policy detail which can straightjacket and limit the richness of models in action. Often so-called neutral or ‘inclusive’ policy prescriptions can mask political forces pushing and pulling definitions in one direction or the other to fit certain stakeholders’ conceptions of what should count as a legitimate wholly online development. When one partial view dominates policy formulation then it can mute other alternative and legitimate conceptions on what might reasonably count as an appropriate environment to carry the spirit of wholly online policy. This can in turn undermine genuinely innovative approaches attuned to the views and needs of academic teaching staff in different disciplines, with different pedagogical orientations and different experiences and values relating to the appropriate use of digital media and online technologies. The cases show that there was resistance to pressures to conform to a certain policy view of the form of online units. At a point this was recognised by the University’s Senior Executive who wished to use the wholly online requirement to foster diversity and creativity of approach. This was observed by one unit team chair participating in the University’s Online Teaching and Learning Fellowship program in 2004: A previous Deputy Vice Chancellor, … said, “You’re here, in a sense, to innovate. We want to move ahead. We want to stretch the envelope. … He said, essentially, “Go for it” and I thought, “Oh great. This is really good.” So he gave me permission as it were, to do that thinking (Interview 12). The second observation relates to the University’s sensitivity to student opinion on having to undertake at least one unit wholly online as part of their undergraduate studies, and more generally amongst staff in relation to the apparent University strategy of wishing to take the institution strongly online. By this we do not suggest a strategy to make the institution an exclusively online University, but to make its competitive strength focussed on online and distance education. From these countervailing 126 Australasian Journal of Educational Technology, 2007, 23(1) pressures, the organisation has now settled on the minimal number of units offered wholly online to provide universal curriculum coverage and reasonable student choice across the undergraduate program. Through these feedback loops the organisation in a sense learned to temper its position in how much and how quickly it could move to online education for all its student cohorts both in relation to the number of wholly online units and other ways in which online supports campus and off campus based modes of enrolment. In this respect, Hagel (2005) has highlighted the importance of study (or delivery) mode in students’ choice of courses, and that students’ have certain expectations of what the mode of study will be as associated with their mode of enrolment, either internal/on campus or external/off campus. With internal/on campus mode of enrolment, students expect interpersonal communication as a key benefit, and with external/off campus enrolment students expect flexibility and independence. Hagel’s analysis sheds light on the students’ experiences of learning in highly online environments. By making all students irrespective of their preferred mode of enrolment/delivery undertake a wholly online unit (an intentionally designed different type of mode of delivery for all student cohorts) evidence of disruption of students’ expectations is evident. For students enrolled internal/on campus the loss is keenly felt in the area of no in person, face to face teaching: I think I would have benefited more from contact learning. I think it is much more interesting to attend a lecture and hear perhaps ideas and experiences from the lecturer. It is also great to have discussions in a tute and bounce ideas off each other. The on-line tests are worth having as they encouraged me to read the chapters every week instead of putting it off until a later date. I have found … [unit title] very interesting but I chose to be an on-campus student and not an off campus student. Personally I did not enjoy the unit being wholly online. Being an On Campus student it was hard for me to get down to reading all the notes required every week. I am used to going to lectures and having the contact, just reading the info is far less fun and much harder. The idea of running a wholly online sounded promising when I first heard it, with the ability to fit around work and social commitments, but in reality it became an after thought when 3 other subjects are pushing for assignments and homework that must be completed before tutorials and labs. On the other hand, while certain off campus students perceive benefit in the quality of the digital resources and online communication channels (‘The quality of material used and available for this wholly online subject has been excellent. I am an off campus student and if all my subjects had this quality of material I would not feel as disadvantaged as I do compared to those students who are able to attend lectures and tutorials’), others Holt and Challis 127 detect the loss of flexibility who have enrolled part time whether they be on campus or off campus: The restrictions placed on completing each module within the given week places undue press on part-time students. I presently work full time and as a result am not able to allocate sufficient time within a given week to cover all subjects. This is further impacted in that I am not always able to undertake my leave during semester breaks therefore am taking time off during the semester. With the current structure of the assessment I could not go anywhere where I could not connect. Furthermore, given that I had to ensure that all assessment was completed within a given week I either had to prioritise this subject over another or not have sufficient time to complete the preferred study before undertaking assessment. I would hope that in future that the assessment could be opened for an extended period to enable a more flexible approach to online study as I would expect the idea behind this study method would encourage [this]. It is not argued that this different learning experience is inherently flawed as proven by students’ views on its immediate value. The institution is attempting through its wholly online requirement to make students more job ready for the demands of contemporary, virtual and distributed workplaces. However, institutional sensitivity to student evaluations, and the inherently partial nature of such evidence gathering, can easily undermine efforts at genuine innovation in teaching and learning. As observed by one wholly online teacher: And it’s unfortunate that the university has this… benchmark of ‘This unit was taught well’ because it’s completely irrelevant to online learning because there’s no definition of taught. Is ‘taught’ someone who actually, you know, gets up and shows you physically how to do something? … The word ‘student’… implies you read and you study and you learn and you’re not, as compared to a pupil, which is someone who looks at a teacher and looks for a teacher to show them the way. And pupils… aren’t tertiary (Interview 1). Moreover, three years on, the definition of wholly online itself has been modified both in response to the diversity of models which have evolved and to the problems which have emerged in taking a strict line in regard to the detailed requirements of the original definition. For example, the new definition recognises that CD or DVD is a legitimate and ongoing platform for delivering digital resources to all students and that while ‘all teaching and administrative support is provided online’ there is now not an explicit requirement for all aspects of assignment processing to take place online or that online discussion will take place weekly or regularly as planned at the beginning of semester and be tied to assessment as appropriate. The new definition merely indicates on the latter that ‘the unit is designed to help students to develop their skills in communicating and collaborating in an online environment’. There is a sense in which the organisation’s own 128 Australasian Journal of Educational Technology, 2007, 23(1) learning experiences have been reasonably taken into account in modifying policy definition to be more inclusive of different models of developing and teaching wholly online units. The change helps to legitimise a broader range of interpretations of what it might mean to operate within the spirit of wholly online and also a growing number of e-learning technologies which might support such an experience. This represents a genuine closing of the loop between policy formulation, implementation, reflection and policy adaptation, to capture the unintended positive consequences of unit development models for wholly online which were not necessarily in accordance with the original definition, and assumptions about the strongly preferred virtual learning technology platform. Conclusion Institutional policy directives do not necessarily lead to straightforward, predictable, and unproblematic teaching and learning outcomes. Transformatory teaching and learning change does not come easily within organisational contexts often constrained by other non-pedagogical considerations. The study has shown how prescriptive policy making can yield a quite unanticipated diversity of models of teaching and learning development in local cultural and political contexts. Again, unpredictably, the emergence of such diversity and accompanying forms of innovation can reflexively act upon original policy directives in ways which reflect organisational learning. Policy may initially shape the realities of practice, and yet the realities of practice can in turn shape policy if the organisation is responsive to its environment. We see that leading and managing such flux is a key to creating overall positive change in the system. This can be generalised about the university’s commitment to the wholly online experience, and from it to other forms of strategic change interventions relating to transformatory teaching and learning in higher education. The ultimate success of any such change agenda will be determined as much, if not more, by the organisation’s perseverance and belief in the values underpinning the initial intervention (what might be seen as the driving vision) as in the prescription of what the detailed outcomes should look like in the policy formulation phase. Beyond prescriptive online technology policy primarily designed to ensure wide spread institutional usage and minimum standards of quality, lies the emerging development relating to conceiving educational environments through the frame of learning experiences design. As descriptors of modes of delivery like ‘open’, ‘distance’, ‘flexible’ and ‘blended’ become more problematic and less helpful in mobilising progressive pedagogies and the proliferation of e-learning technologies, higher education environments will be increasingly informed by approaches which design and align desired learning experiences with the various circumstances of diverse Holt and Challis 129 student cohorts. E-learning technologies will be brought into service of these learning experience designs in customised and personalised ways, and in appropriate balance with various forms of face to face teaching/learning. The need for institutional policy to mandate different levels and uses of online technologies is therefore unlikely to be permanent. These imperatives will begin to fade in the face of the movement to designs of valued learning experiences and are increasingly being informed by student feedback. Ultimately, the ‘e’ in e-learning and its policy directives will recede from the higher education landscape. We caution, though, that policy makers may lag in their appreciation of this emerging development with a new set of innovative learning experience design practices likely to place further pressure on existing policy frameworks. Endnote 1. Until 1999, Di Challis published under the name of Diane J. Thompson. References Armatas, C., Holt, D. & Rice, M. (2004). Designing distributed learning environments in support of professional development in the field of psychology. Educational Media International, 41(4), 315-26. Biggs, J. (2003). Teaching for Quality Learning at University What the student does. Buckingham: SRHE & Open University Press. Bonk, C. (2004). The Perfect E-Storm: Emerging technology, enormous learner demand, enhanced pedagogy, and erased budgets. The Observatory of Borderless Education. http://www.publicationshare.com/part1.pdf [viewed 11 June 2005] Candy, P., Crebert, G. & O’Leary, J. (1994). Developing Lifelong Learners through Undergraduate Education. Canberra: AGPS. [verified 20 Feb 2007] http://www.dest.gov.au/sectors/training_skills/publications_resources/prof iles/nbeet/hec/developing_lifelong_learners_through_undergraduate.htm Carr, W. & Kemmis, S. (1986). Becoming Critical: Education, Knowledge and Action Research. Geelong: Deakin University Press. Challis, D. (2005). Eroding distinctiveness: Blurring the boundaries between on- and off-campus students by the adoption of learning management systems. Paper presented at the ODLAA Conference, Adelaide, 8-11 November. [verified 20 Feb 2007} http://www.odlaa.org/events/2005conf/ref/ODLAA2005Challis.pdf Challis, D., Holt, D. & Rice, M. (2005). Staff perceptions of the role of technology in experiential learning: A case study from an Australian university. Australasian Journal of Educational Technology, 21(1), 19-39. http://www.ascilite.org.au/ajet/ajet21/challis.html 130 Australasian Journal of Educational Technology, 2007, 23(1) Coates, H. (2005). Leveraging LMSs to enhance campus-based student engagement, EDUCAUSE Quarterly, 28(1). [viewed 6 July 2005] http://www.educause.edu/apps/eq/eqm05/eqm05110.asp CSTL Item Bank. [viewed 29 April 2005 at http://cstl.syr.edu/Cstl?T- L/IBank_Q.htm, verified 20 Feb 2007 at http://cstl.syr.edu/cstl2/Home/ Teaching%20Support/Teaching%20at%20SU/Student%20Ratings/12A310.htm DEST (Department of Education, Science and Training) (2002). Universities online: A survey of online education and services in Australia. [viewed 23 May 2006] http://www.dest.gov.au/NR/rdonlyres/69BC95EB-CAE4-4A4B-80DE- C74BB5F33E10/896/02_a.pdf DEST (Department of Education, Science and Training) (2003). Online Teaching and Learning in Higher Education: A Case Study. [viewed 16 Jan 2006] http://www.dest.gov.au/sectors/higher_education/publications_resources/ other_publications/online_teaching_and_learning_in_higher_education.htm ETL Project (undated). Enhancing Teaching-Learning Environments in Undergraduate Courses. TLA Centre, University of Edinburgh. http://www.ed.ac.uk/etl/ [viewed 2 April 2005, verified 20 Feb 2007] European Liaison Committee (2004). Distance Learning and eLearning in European Policy and Practice: The Vision and the Reality. [viewed 11 January 2006] http://www.odl-liaison.org/pages.php?PN=policy-paper_2004 Hagel, P. (2005). Study Mode and Student University Choice: An Empirical Investigation. Unpublished PhD thesis. Holt, D. (1993). Changing conceptions and practices of management: Professional learning from an MBA experience by distance education. Distance Education, 14(2), 232-59. Holt, D. & Thompson, D. J. (1995). Responding to the technological imperative: The experience of one open and distance education institution. Distance Education, 16(1), 43-64. Holt, D. & Thompson, D.J. (1998). Managing information technology in open and distance higher education. Distance Education, 19(2), 197-227. Holt, D., Rice, M., Segrave, S. & Thompson, D. (1997). The case of the case: The interplay between research and practice on the web. In T. Evans, D. Thompson and V. Jakupec (Eds), Research In Distance Education 4, Revised papers from fourth Research in Distance Education conference, Deakin University Press, Deakin University, 18-33. Holt, D., Mackay, D. & Smith, R. (2004). Developing professional expertise in the knowledge economy: Integrating industry-based learning with the academic curriculum in the field of information technology. Asia-Pacific Journal of Cooperative Education, 5(2), 1-11. Holt and Challis 131 Segrave, S., Holt, D. M. & Farmer, J. (2005). The power of the 6three model for enhancing capacities for effective online teaching and learning: Benefits, initiatives and future directions. Australasian Journal of Educational Technology, 21(1), 118-135. http://www.ascilite.org.au/ajet/ajet21/segrave.html Smith, R., Mackay, D., Challis, D. & Holt, D. (2006). Seeking industry perspectives to enhance experiential learning in university-industry partnerships: Going beyond mere assumptions, Asia-Pacific Journal of Cooperative Education, 7(2), 1-9. Southwell, D., Gannaway, D., Orrell, J., Chalmers, D. & Abraham, C. (2005). Strategies for effective dissemination of project outcomes. A report for the Carrick Institute for Learning and Teaching in Higher Education. Commonwealth of Australia. Thompson, D. & Holt, D. (1996). Tertiary pedagogy encounters the technological imperative. Distance Education, 17(2), 335-54. Thompson, D. (1996). The tape-recorder as a mediating factor in research. Australian Educational Researcher, 23(3), 1-13. Walker, R. (1985). Doing Research A Handbook for Teachers. London: Methuen & Co. Dr Dale M. Holt, Head, Educational Design, Professional Development and Research, Institute of Teaching and Learning, Deakin University, Geelong Victoria 3217, Australia. Email: dale.holt@deakin.edu.au Dr Di Challis, Institute of Teaching and Learning, Deakin University, Geelong Victoria 3217, Australia. (Until 1999, Di Challis published under the name of Diane J. Thompson). Email: di.challis@gmail.com