wozniak.pdf Australasian Journal of Educational Technology 2012, 28(5), 896-911 Design-based research principles for student orientation to online study: Capturing the lessons learnt Helen Wozniak Flinders University Jenny Pizzica University of Technology, Sydney Mary Jane Mahony The University of Sydney Few institutions have reported research on students' use of orientation programs designed for mature students returning to study in contemporary learning environments now regularly amalgamating distance and online strategies. We report within a design-based research framework the student experience of GetLearning, the third stage of an innovative online orientation program. Integral to the design is active student engagement with activities to extend their skills whilst still supported in a risk free environment. Analysis included observations of the patterns of student activity, students’ responses in evaluation instruments, our reflections, and identification of the limitations to deployment of such orientation programs. We also provide a set of design principles to guide further development work and research. Introduction An orientation program is commonly recognised as an appropriate institutional provision for new university students. With the increasing use of an amalgam of online and distance learning approaches, particularly in programs targeted for mature age students, there is a need to rethink how universities approach the orientation process. These students are likely to be inexperienced in the distance education necessities of independence, self-regulation and coping with the perceived isolation of studying off campus. At the same time they may not be substantially experienced in using a computer-supported environment for general study and participation in a learning community. This paper reports findings and discusses the lessons learned in a design-based research study of student experience during the first implementation year of GetLearning, the third stage of an innovatively conceptualised orientation strategy (known as GettingOnTrack) for such students. Background Descriptions of formal online learning induction for learners new to using online technology are sparse. Information provided to students usually focuses on enhancing their computer skills, demonstrations showing how to navigate around course management systems, and text heavy lists of FAQs and other information (Ko & Wozniak, Pizzica and Mahony 897 Rossen, 2010). Levy (2006) is one exception, suggesting that orientation should form part of a broader introduction to not only the learning space, but also the information environment and pedagogical approach to studying online. She notes that the failure of students to engage early in their studies often disadvantages them due to their lack of awareness of the resources available to assist them and their inability to fully commit to early discussion activities. Instead they are distracted by the vastness of the learning environment. More recently Australian research also indicates that although current students may have been engaged in using technology from an early age, they may not necessarily understand how to use the technology for learning (Kennedy, Judd, Churchward, Gray & Krause, 2008). Literature describing the student experience of beginning online study (Levy, 2006; Motteram & Forrester, 2005; Price, Richardson & Jelfs, 2007), substantial local anecdotes and our own research (Wozniak, Mahony, Pizzica & Koulias, 2007) all indicate that pre-semester orientation as well as support during the semester is required. As Salmon (1998) indicated in her early work, there is a need for learners to be provided with opportunities to experiment with the technology and to make mistakes in a supported environment. Parallel investigations have also been directed at determining the factors that contribute to student satisfaction with online learning. Elements that have been identified as influencing student satisfaction include ease of use of the online system, learner anxiety with the technology, perceived usefulness of the materials, instructor attitudes, course design and flexibility (Chang & Tung, 2008; Sun, Tsai, Finger, Chen & Yeh, 2008). Reports that describe orientation initiatives for online learners (Bozarth, Chapman & LaMonica, 2004; Levy, 2006; Scagnoli, 2001; Tomei, Hagel, Rineer, Mastandrea & Scolon, 2009), note the achievement of increased student enjoyment and confidence. However with the exception of Levy (2006), they retain a sense of atomised, superficial and rather piecemeal strategies without a coherent underpinning conceptual framework for delivering such improvements. Instead, what is often recommended are resources to improve computer skills or a review of static information loaded webpages without consideration as to how this is best achieved. However, the 2010 release of the Student Induction to E-Learning (SIEL) report (IMS Global Learning Consortium Inc, 2010; Krause & McEwen, 2009) recognised the need for best practice frameworks to guide student induction to e-learning. This report provides a checklist and maps a best practice framework for promoting post-secondary student retention associated with induction to e-learning. It suggests the need for a pro-active approach to orientation. Although this guidance undoubtedly can contribute to improved practice, there remains a need to examine more closely the student experience of this crucial stage which lays the foundation for successful study in the following years (Crosling & Heagney, 2009), echoing the suggestion made earlier by Motteram and Forrester (2005). ... we consider that more student-centred studies exploring online distance learning are needed to assess how students manage the transition to online learning, how they perceive their initial online tasks, what strategies (if any) they adopt, and how the appropriate use of technology and pedagogy could enhance distance programmes, benefit students’ learning, and enrich their educational experience (p295) 898 Australasian Journal of Educational Technology, 2012, 28(5) Our research describes the implementation of one such approach known as the GettingOnTrack suite of resources founded on a simple, yet effective conceptual framework (Wozniak, Mahony, Lever & Pizzica, 2009). A design-based research framework The research reported here is situated within the design-based research framework (Collins, Joseph & Bielaczyc, 2004). This is an ideal approach for investigating complex and real world educational problems as it enables researchers and practitioners to collaborate through often long term research cycles of analysis, development, evaluation and reflection (Reeves, Herrington & Oliver, 2005; Reeves, McKenney & Herrington, 2011). The intention of this work was a multi-dimensional approach to research about educational design, and could be described in the words of Zaritsky, Kelly, Flowers, Rogers and O’Neill (2003) as: Implicitly and explicitly directed at creating products and processes for the improvement of student learning, teacher skills, and institutional and systemic reform (p.32). This paper is structured into five sections as recommended by Collins, Joseph and Bielaczyc (2004) for reporting design-based research: • Setting where the design was implemented, • Underpinning goals and design principles that form the basis of the solution developed, • Description of each iteration so that the evolution of the design is captured as well as the reasons why changes were made to the design • Outcomes found as a result of the implementation of the design including a wide variety of both qualitative and quantitative data. • Lessons learned which emerge from the evolution of design principles and which contribute to further knowledge and theories as well as guiding future researchers about the limitations of the design. Three iterations of implementation were undertaken: • Iteration 1: review by expert teachers (Semester 2, 2006) • Iteration 2: released to students and for review by an experienced student (Semester 1, 2007) • Iteration 3: released to an increased range of student cohorts (Semester 2, 2007) This paper primarily focuses on iterations 2 and 3, with a brief report of iteration 1 included in the next section; the setting. The setting Before we consider the goals and elements of the design developed for this design- based research, it is important to contextualise the setting. The investigation was conducted in a large, research-intensive, campus-focused Australian university. A new conceptual framework for orientation of mature age students returning to study in a primarily or wholly distance and online mode was developed and put into operation to support postgraduate students in the human health sciences. Implemented in 2007, the full orientation program is GettingOnTrack, a suite of engaging and flexible learning resources comprising three interrelated parts, GetReal, GetStarted and Wozniak, Pizzica and Mahony 899 GetLearning. The aim was to enable these students to prepare themselves for the distance education or blended learning delivery environment within a suitable timeframe. See Wozniak, Mahony, Lever and Pizzica (2009) for a more detailed description; a brief overview is provided here. GettingOnTrack is designed to begin prior to enrolment (and even admission) via the GetReal open access, web-based interface (Lever, Mahony & Wozniak, 2007). Prospective students can assess their readiness for study by comparing course design attributes with their perceived needs, gauge their technological capabilities, and engage in a time management task. After admission and enrolment in their initial subjects, students are provided with a simple guide (GetStarted) to assist them to navigate and log into the learning management system (LMS) site. They are then able to access GetLearning, housed in the university’s enterprise LMS, where they encounter a modularised introduction to key elements of learning in online environments. The focus in this paper is on the third component, GetLearning. This suite of resources was designed in keeping with the emerging conceptual framework (Wozniak et al., 2009). Design and content drew on prior work carried out with undergraduate students in blended learning environments evaluating the effectiveness of activity based online communication tasks (Wozniak, 2007; Wozniak & Silveira, 2008), and on other activities piloted in a range of faculties across the university (Freeman, Clarkeburn & Treleaven, 2007). Iteration 1 encompassed a comprehensive expert teacher evaluation. Reviewers included local champions of online learning and over twenty teaching and support staff from relevant postgraduate degree programs (at least some of whom were targeted stakeholders for the new orientation resource). A proforma reporting sheet was used and more than twenty pages of comments (both general and specific) were received, enabling further refinement. The broad design concept was confirmed, with no major design faults. Underpinning goals and design principles The learning design strategy underlying GettingOnTrack as a whole is one of reducing students’ transition shock from their current world (often involving work and family life) to an academic world of study online, by spreading the cognitive load in each of three key dimensions: reflective, interpersonal and technical, whilst managing the disparities that arise at different stages in this transition process (Lever et al., 2007). G e t L e a r n i n g provides a gradual post-enrolment initiation into the role and responsibilities of the online learner in a university learning environment. A guiding design principle was that orientation success would be best achieved if the student can participate directly in activities within a safe risk free environment. Consequently, there is a broadly sequential design starting with simple navigation exercises, moving through communicating with others online and working in collaborative online groups, continuing to essentials such as submitting assignments online, understanding relevant university policies and accessing information sources, and avoiding plagiarism. The activities are learner-centred, self-motivating, low maintenance and applicable to different course contexts. Further discussion of the underlying design principles are explained in Wozniak et al (2009). Also included is an overview of the five modules contained in the GetLearning resource including the types of interactive activities and the supports available to assist new online learners as they progress through each module. 900 Australasian Journal of Educational Technology, 2012, 28(5) The main study: Iterations 2 and 3 Students in ten programs across three of the University’s 16 faculties were provided with access to GetLearning in 2007. In Semester 1, 179 postgraduate students enrolled in five coursework Master’s programs in the Faculties of Health Sciences and Medicine and a professional doctorate program in the Faculty of Health Sciences. In Semester 2 the cohort was expanded to include 292 students enrolled in post graduate studies in a third faculty: Nursing and Midwifery. Overall, the majority of students were commencing their studies, and ranged widely in age (from early 20s to late 60s). Approval was gained from the University’s Human Research Ethics Committee. Data sources included a wide variety of both quantitative and qualitative types collected across two academic semesters from February 2007 to November 2007. This included: • Website usage statistics • Tracking data for individual students collected by the LMS • Moderators’ journals during iteration 2. • Anonymous ‘3 minute feedback’ for each of the five modules (completed online at the end of each module asking about the achievement of the module specific learning objectives, and open ended responses to questions about the relevance and helpfulness of the module for their learning In addition, the research design originally included focus groups with a selection of students and an inquiry into the possible impact of the orientation experience on subsequent academic achievement. The site was made available to postgraduate students prior to the commencement of each of two semesters in 2007 (one week prior for semester 1 and two weeks prior for semester 2). Students were sent an advisory email by a teacher responsible for the unit or course in which they were enrolled. They were provided with the GetStarted guide in PDF or hardcopy, and encouraged to work through the five modules, commencing immediately and recommended to be finished by the end of the semester’s first week. Their participation in the GetLearning site was not mandated and did not contribute to assessment scores. On entry into the site they received an announcement about the research and were asked to indicate if they would accept an invitation to participate in a focus group session. Student response to this was insufficient and focus groups were not conducted (see the section ‘Lessons’ for further discussion). Students were asked to complete an anonymous '3 minute feedback' evaluation survey at the completion of each module. Moderators’ journals During the iteration 2, two of the authors closely moderated the discussion board activities (three activities in total, occurring in modules 2 and 3) and on a daily basis monitored student use of the site for the week prior to the formal commencement of the semester and through the first three weeks of the semester. They provided feedback and suggestions to the students who had accessed but missed key areas or who had not accessed the site. They also kept reflective diaries during the first month of the semester, recording their time involvement, their actions and their observations of student activity. In contrast, during the second semester there was no ongoing support provided to students, and no ongoing moderation of the discussion board activities except for a welcome posting made by the moderator. Wozniak, Pizzica and Mahony 901 This approach allowed the moderators and designers of the site to compare their expectations of how students would use the site with how students did use the site. In combination, these evaluation methods allowed the moderators to act upon the students’ patterns, providing further targeted support to students. For Iteration 3, the moderation activities were reviewed and a ‘minimalist model’ developed whereby students were not prompted to explore additional modules in the site. This was a pragmatic response to limited resources as well as an opportunity to observe student activity under limited moderation conditions. LMS tracking data During both semesters detailed tracking data for individual students was reviewed to provide information about the timing of student access, pattern of access (the order of pages, how they moved through the site and modules) and the degree of interactivity and the type of activities completed. Data were collected by downloading tracking statistics and survey responses. SPSS was used to provide descriptive statistics and all open ended answers were read by all researchers and grouped according to thematic emphasis. Patterns of access were coded by analysing the tracking sequences taken by students. As noted above during Iteration 2 in Semester 1, students were given further advice and support on how to better utilise the module resources, and tracking data was again captured to determine the impact of the intervention suggestion provided by the moderator. The effect of these interventions on participation during iteration 2 has been reported previously (Wozniak et al., 2007). Accessing the site and completing the activities in each module was recommended to students by their course coordinators, but was not mandated or assessed. In Semester 1, 79% of students (n=142) accessed the site within the first three weeks of semester. In Semester 2, 69% (n=202) accessed the site in the same timeframe. Outcomes from the implementation Although flexibility was a design element, the modules were still prepared, in the minds of the designers, as a logically linear sequence, with the expectation that students would work through each module, commencing with module 1 and continuing through each subsequent module until completion. Tracking data indicated that few students followed this linear pattern. Participation patterns Students fell into four groups according to how they first accessed the site, detailed in Table 1. It was anticipated that the pattern of initial access shown by group 1 would be the most common with students selecting one of the 5 modules from the homepage. This was certainly the case in semester 1 with module 1 ”Finding your way around” (n=51) and module 4 “Getting your assignment done” (n=13) the most commonly selected module for first review. This pattern was much less common in semester 2 with the most popular modules again being module 1 and 4 (n=21, n=13 respectively). 902 Australasian Journal of Educational Technology, 2012, 28(5) Table 1: Access patterns and completion patterns noted from the first access to site % (no.) Group Identifiable orientationpattern from first access Semester 1n=179 Semester 2 n=292 1. Accessed modules Selected one of the 5 modules from the homepage 45% (81) 14% (42) 2. Accessed homepage only Accessed the homepage but did not explore further during this initial access 23% (41) 15% (43) 3. Accessed site but not via the homepage Accessed site from icons or course tools that lead them directly to announcements, or other area such as a discussion posting rather than through the homepage 20% (35) 48% (141) 4. No access Students never accessed the site 12% (22) 23% (66) The second pattern, demonstrated by group 2, accessing the homepage without any further access to deeper pages on the first visit, was also a common pattern. Of greater significance is determining if this group returned to the site at a later date. In semester 1, 28 of the 41 students (68%) returned to the site from a few hours to 10 weeks later, and likewise 24 of the 43 (56%) in semester 2. This may illustrate the need for a clear stimulus to assist the learner to go beyond the initial browse of the homepage. Another pattern (Group 3), not initially expected, occurred when students navigated on their first access directly to the embedded activities and tools such as the discussions, mail, assignments, and calendar and missed the homepage and content of the modules. This occurred when they clicked on the icons below the GetLearning title link rather than the title link which would take them to the homepage. A far greater number of students explored the site in this manner in semester 2 possibly because they had become accustomed to this approach from their earlier studies online in semester 1. The main limitation to this type of access on the first visit to the site is that students miss the context of the site. For example, going directly to the discussion board without reviewing the activity information may confuse the student by not locating the activity in the context of development of online communication skills (module 2). Of the students who accessed the site in this manner a proportion did return to the site to access the modules at a later date (21 of 35 (60%) in semester 1 and 57 of 141 (40%) in semester 2). Table 2 outlines the overall type of participation in the GetLearning resource for all students who accessed the site during iterations 2 and 3. There is a marked difference in participation between each semester with greater access to the module content and greater participation in all the activities during semester 1. During semester 2 there was a greater student 'lurking' role in the discussion board where students were noted as reading the discussion board postings rather than engaging in active participation. It was noted that if students did access the module they then navigated to deep pages within that modular content area, but this did not always lead to active participation in the related activity (discussion, quiz or assignment submission). Findings also indicated that students returned to GetLearning throughout the semester, multiple times. In semester 1, 25% of students accessed the site more than 10 times during the semester. This was considerably less in semester 2 (8%). However in both semesters students continued to access the site after the conclusion of semester teaching periods (25 students in semester 1 and 23 in semester 2), presumably to Wozniak, Pizzica and Mahony 903 review different elements as the need arose. Exploration of this unexpected outcome is outside the scope of this paper, with further investigation of student use of orientation resources over time being warranted. Table 2: Modules accessed and activity completions Semester 1 n=157 Semester 2 n=226 Accessed modules regardless of their entry route into the site 119 108Modules Accessed by any means but didn’t review any modules 38 118 Read discussion postings but did not post any messages 32 51 Read and posted a discussion item 62 30 Started and completed a quiz activity 74 27 Completed activities Started and completed an assignment submission 30 10 3-minute feedback Completion of the module feedback surveys ranged from 41 to 5 responses across all the modules with a tendency for a reduction in completions across the site and in Semester 2 as detailed in Table 3. As the surveys were completed anonymously, it is likely that the same student completed more than one survey. Table 3: Completion of module feedback surveys Module number: Content focus Semester 1 n=119 students who accessed one or more modules Semester 2 n=108 Module 1: Navigation 40 27 Module 2: Communication 41 21 Module 3: Groups 28 5 Module 4: Assignments 29 8 Module 5: Academic integrity 12 5 Total responses for all surveys 150 66 Analysis of the survey results indicated that participants agreed or strongly agreed that the modules were helpful (Semester 1, 96.7%, Semester 2, 80.3%) and relevant (Semester 1, 88.9%; semester 2, 81.8%). Respondents were also asked to self-assess if they felt the module objectives had been accomplished. Combining these responses across all modules indicated that in Semester 1, 78% (n=117) and in Semester 2, 62.1% (n=41) felt they had met all the objectives in the modules. Further, in Semester 1, 11.3% (n=17) and in Semester 2, 21.2% (n=14) felt they had met one or more of the objectives. In open-ended questions participants were invited to comment on what worked well, why it worked, what needed improvement and what further assistance they felt they needed. Comments were grouped into the following themes. Practising activities Many participants commented on the “hands on” and “practical” approach of the activities in the site. The instructions and guidelines for activities in each module were described as “well structured”, “easy to read” and “easy to follow”. Importantly, students felt they were “not overwhelmed with too many instructions” and actively participated in the activities embedded in each module. Participants reported active behaviours such as “clicking around”, “practising”, “experimenting”, “making 904 Australasian Journal of Educational Technology, 2012, 28(5) mistakes and correcting them”, “flipping through the windows”, “enacting the tasks” and referring to the instructions “repeatedly”. Perceived relevance Participants expressed the perceived relevance of the site in a variety of ways. They saw the activities in the site as both “real” and “enabling”. The use of real discussion groups drew many positive comments as encouraging participation “beyond the simple posting of singular messages” and providing support and encouragement such as seeing “what other people had to say regarding the problems they were having also”. In activities such as submission of assignments, participants noted how closely the practice task mirrored the requirements of assessment tasks within the courses in which they were enrolled. In activities such as managing email accounts, searching electronic databases and locating and downloading electronic journal articles through the library, participants expressed enthusiasm and relief at being able to see the immediate results of their efforts and being guided to successfully complete tasks which had previously been a “source of intimidation and concern”. Only one participant indicated that they would have found the site better if it had been made available only to students within the same course. A safe environment Aligned to participants’ valuing the relevance and the opportunity to practise, many also enthusiastically expressed the value of doing so in an environment they considered to be “non-threatening”. The opportunity for practice activities was seen as “confidence building” and allaying “anxiety”. Timing The timing of access to the resource was mentioned by both Semester 1 and Semester 2 participants. Continuing students offered the resource in Semester 2 indicated that, although the resource was still useful, access to it in the first semester of their enrolment would have been preferable. I found that the example questions worked well as they enabled a better insight into the module. It is a shame this approach to course requirements was not incorporated at the beginning of the first semester. It came a bit late should have been introduced on the first semester on my part. But it is good tool. Commencing students also indicated that earlier access to the resource would have been helpful (timely availability is addressed in the Lessons section below) I felt I should have received this much earlier, especially when I was accepted on the course, now I feel am rushing this, when there is a lot of other things that have to be taken into consideration. I think this is a brilliant idea & is good for those who haven’t studied in ages Most things worked well but I could not log into any of these activities until the 6th of March which made completing them before the start impossible. I was looking daily… The lessons learnt and the emergent design principles The findings presented above indicate that GetLearning (and indeed the full suite, GettingOnTrack) beneficially addressed the call for commencing student orientation at Wozniak, Pizzica and Mahony 905 the postgraduate level, not only for speeding up the transition to online learning, but also providing authentic tasks in a risk free environment. Extra time at the start would have saved me hours later on. It works well because it is like a practice run to the real thing. Examining the initial student access patterns to GetLearning has helped to identify the most pertinent types of interventions that can assist the student’s entry to the real online learning environment. Individualised supports and feedback can assist students to make the transition to online learning and feel comfortable about participating in online learning communities. GettingOnTrack models these interventions and provides advice to other academics considering how they might incorporate this into their own context. At the same time this research made visible some conundrums for student support and for research into its impact, with factors that promote student engagement potentially at odds with institutional constraints and the culture of academic autonomy. In addition there are continuing challenges for conducting useful research with busy students. Student engagement Encouraging students to access and progress through the module content, and also participate actively in the related tasks is a key challenge. The varied access patterns that we found concur with the findings of Kehrwald et al (2011), showing that learners’ needs are diverse. O’Donnell et al (2006) evaluated a two-stage package to introduce both online and on campus students to online study at the University of Ulster. Despite all students being provided with access to the resource housed within their LMS, only approximately 20% of students logged into the package, with even less (10%) reviewing more than 10 pages of content. Their students completed a series of quizzes related to the content of each component of the package, commenting in the evaluation survey that the package had equipped them with the knowledge and skills to study online. Interestingly, one item in the evaluation that received a lower agreement from the students (58% strongly agreeing or agreeing) was their desire to try out using communication tools to communicate with other students also participating in the induction package. This result contrasts with our findings showing that even if students didn’t actively participate in communication activities, they did access these areas, engage in lurking behaviours, and report positively about this experience. Clearly there is a need to more specifically highlight the benefits gained from collaborative online learning to beginning students. How and when to achieve this is open to debate: is this best fostered as part of a generic orientation resource or whilst engaged in activities in a disciplinary context? Timeliness GetLearning was designed for students to commence at the beginning of orientation week (one week before the formal start of teaching) and to complete within the first three weeks of the semester. This reflected known institutional constraints, and even so could not be met for all students. Administrative procedures meant that although students could be admitted to courses long before the semester commenced, they were not formally enrolled as students for the semester until close to the beginning of, during orientation week, or even after the semester had begun. This appears to be a 906 Australasian Journal of Educational Technology, 2012, 28(5) result both of an on-campus culture and administrative batch processing. The on- campus culture reinforces an expectation that commencing students would enrol in their subjects during orientation week, or even after the commencement of the semester’s formal teaching. The Student Induction to E-Learning report (IMS Global Learning Consortium Inc, 2010) highlights the need for a sustained and multifaceted approach, beginning prior to enrolment, as well as during the early stages of study, recommending guidance and support for learner centred activities as well as the development of remediation action plans for students at risk of withdrawing from study. The concept of 'just in time' support and incremental development of skills has been recommended as a key issue requiring further investigation (Forrester & Motteram, 2005). There is an emerging trend towards universities providing opportunities for new students to engage more actively with the university culture. One example is the University of Western Australia’s Facebook page which is directing its support to students in the pre- enrolment period (Cluett, 2010) – such an approach circumvents institutional administrative limitations, but also potentially takes students into less secure territory where issues of digital identity become more apparent (Wozniak, Uys & Mahony, 2012). Moderation The observed student use patterns reinforced the need for timely, interaction-based orientation activities, and highlighted students’ need for both individual support and ongoing access to orientation style resources throughout the semester. Moderation of discussion boards is highly visible in the burgeoning literature on online learning, but tends to be limited to online discussions. Moderation in our view is a broader role, described as both risk management for individual students and establishing institutional and limited teacher presence (Garrison & Anderson, 2003). Our findings indicated the value of using LMS tracking data to identify students who have not entered the site (inactive), who appear to have lost their way in its technicalities, and/or who have chosen a pathway to suit their own perception of their needs with attendant risks to being ill-prepared in the areas skipped. This was the most concerning for Semester 2 where moderation did not occur, and about 50% of students did not access any of the module content. Perhaps this was because in Semester 2 students were not new to online study but perhaps they needed greater scaffolding and support. In the case of GetLearning, our particular concern was for those students who did not engage in the discussion board activities included in modules 2 and 3. Moderation of discussion board in a minimalist but visible presence is necessary, not only for the activities recommended by Salmon (2000), but to contribute to the sense of institutional teacher presence (Shin, 2003) for students new to the institution. Integration with formal study On the face of it, closer integration of orientation activities with the students’ program of study should be recommended. This would importantly raise the profile of managing study time and improve the motivation required to complete the activities and should additionally increase teacher presence. O’Donnell et al (2006) concluded that their induction package should be further implemented as a template-driven model, modified according to the needs of specific population cohorts. This was seen as their next development step which would enable inclusion of communication based Wozniak, Pizzica and Mahony 907 activities managed by local tutors, and also overcome their difficulty in managing a large cohort of students centrally. On the other hand, however, integration with formal study may restrict the opportunity for students to stumble around, ask ‘stupid’ questions and make mistakes in an environment outside their formal study space where the student may view such actions as having higher stakes in terms of personal status and perception. There is also the issue of workload. The 2007 GetLearning delivery was an effort across a number of academic programs in order to manage and minimise additional load on already overloaded program delivery staff . Coupled with this dilemma is the consideration as to whether to mandate participation in such resources. Recently Carruth et al (2010) described an online course for graduate nursing students which over time, has become a compulsory component for entry of all new masters level students. Students receive continuing education credits for completion, which contributes to their nursing registration (a reward for learning activity which may be perceived by students as additional to their formal coursework). Requirements include participation in discussion forums with feedback provided via an assessment rubric, completion of a quiz as well as engaging in other collaborative activities such as using a wiki. These authors also note the advantage of students experiencing and participating in a non-threatening online learning environment with survey evaluations reinforcing the value of practising “being an online learner” as a way to reduce the anxiety and apprehension felt by commencing students. Ownership The four concerns set out above lead to the issue of ‘ownership’ of orientation resources. Subjects (units of study in an award course program) are culturally ‘owned’ by the program or indeed the individual lecturer/s in charge of delivering them, at the University of Sydney. Without clear ownership, the ongoing development and maintenance of orientation style resources with embedded activities can be problematic. This is possibly a reason why so many orientation resources do not move beyond a static information delivery stage or are not sustained after the innovator moves away from the institution. Here again lies a point of contention in the limited literature available. Authors (Motteram & Forrester, 2005; ODonnell, 2006; Crosling & Heagney, 2009) note the need for an integrated university wide approach to inducting new students to the university culture, where skills required to learn online are combined with other study skills such as information literacy, academic writing, time management and pastoral care. Yet these same authors consider that this support is best achieved when these activities are embedded in the disciplinary context and are conceived as an ongoing process, perhaps recognising the difficulty in coordinating such activities across a large number of stakeholders in any one university. Limitations to the research How do we encourage students to participate in data gathering for improvement when they have other, better things to do? This study was affected by low participation rates in the 3 minute feedback. Further, the planned focus groups did not go ahead as volunteers to participate were too few in number. We can recommend in principle that strategies for data gathering must be built into the system (hence the LMS tracking data was useful, though of course also limited by its mechanical nature and inbuilt limitations). This can more easily be done if such resources are built into a formal program of study and perhaps tied to assessment. This strategy, however, would be in 908 Australasian Journal of Educational Technology, 2012, 28(5) opposition to our finding of the value of having orientation activities available over a period of time prior to the semester. Emergent design principles and guidelines for future implementation As outlined recently by Reeves et al (2011), as well as developing creative approaches to solving real world educational problems, design-based research has the dual role of developing principles to guide future educational developments. The following design principles are recommended from this research: • Orientation to online learning environments involves a complex interplay of three dimensions - technological, interpersonal and reflective practice - that blend in the transition from a student’s current world, often involving work and family life, to an academic world of study. Designs of orientation resources, therefore, need to provide sufficient triggers to engage students in discovering how this journey will impact their development as learners. Failure to engage students early in this transition may lead to students allowing this opportunity to pass them by. • Orientation is a process that extends over time and continues long after the commencement of semester. Students will tend to access resources as they are needed and may return to review them several times. • The relationship between providing a 'risk free playground' for students to try out using skills required for online learning and embedding such activities within formal discipline based study needs to be considered. The former requires clear ownership for provision of student support and ongoing maintenance, whilst the latter may restrict student activity, due to fear of failure if mistakes are made in a formally assessment environment, and increase workloads for discipline based staff. • Clear navigation is essential, including prompts and supports that acknowledge non-linear pathways of student access. Adequate support that guides students from content to activities and vice versa is essential to fully engage students. • Teacher presence in communication activities appears to be an important factor influencing the motivation for students to participate actively, rather than merely lurk in the background. Recommendations for further research The experience and findings of this study suggest several directions for further research. These include construction of research designs which can compare the impact of moderated and non-moderated delivery strategies, a consideration of institutional pilots that will enable early enrolment of students returning to study which provides access to orientation activities over a longer period prior to the beginning of their formal study period. In addition, we recommend more proactive strategies, such as providing rewards or incentives for students to complete survey data and participate in research using qualitative strategies to enable a deeper understanding of the student journey. Finally, it is important to determine if there is a link between readiness for online study and student satisfaction and success. Wozniak, Pizzica and Mahony 909 Conclusion Orientation for students returning to study in an online or distance learning environment for the majority or all of a program is widely recommended, yet research into the student experience of such programs has been sparse. This study has highlighted the benefits of conducting such research and attempting to move beyond mere student satisfaction surveys. Students, although not all students, take up independent and self-paced orientation opportunities when available, and make use of them in patterns which need deeper investigation to ensure such resources are appropriately designed. Such research, however, will require creative approaches to motivating student participation. Two particular findings of this study are relevant for institutions planning or revising orientation programs for the online/distance world. The first is the challenge of time, and how to enable (and encourage) students to take up the orientation opportunity, not just at the commencement of their studies, but over a period of time which will benefit them. The second is the importance of broader moderation in regard to encouraging students to participate, supporting students who appear to have become lost in the technicalities of the system, and providing minimal but sufficient visible moderation of discussions to ensure they are viewed as authentic and institutionally supported. Acknowledgments The GettingOnTrack development project was supported by the University of Sydney’s eLearning Strategic Initiative, under the auspices of the Pro Vice Chancellor (Learning & Teaching). Research on the project was supported by a University of Sydney 2007 Teaching Improvement & Equipment Grant. We acknowledge the feedback provided by numerous colleagues as well as reviewer comments and Amy DeLaroche’s attention to detail during the analysis of endless LMS tracking data. References Bozarth, J., Chapman, D. D. & LaMonica, L. (2004). Preparing for distance learning: Designing an online student orientation course. Educational Technology & Society, 7(1), 87-106. http://www.ifets.info/journals/7_1/10.pdf Carruth, A. K., Broussard, .P. C., Waldmeier, V. P., Gauthier, D. M. & Mixon, G. (2010). Graduate nursing online orientation course: Transitioning for success. Journal of Nursing Education, 49(12), 687-690. http://dx.doi.org/10.3928/01484834-20100831-06 Chang, S. C. & Tung, F. C. (2008). An empirical investigation of students' behavioural intentions to use the online learning course websites. British Journal of Educational Technology, 39(1), 71- 83. http://dx.doi.org/10.1111/j.1467-8535.2007.00742.x Cluett, L. (2010). Online social networking for outreach, engagement and community: The UWA Students' Facebook page. In Educating for sustainability. Proceedings of the 19th Annual Teaching Learning Forum, 28-29 January 2010. Perth: Edith Cowan University. http://otl.curtin.edu.au/tlf/tlf2010/refereed/cluett.html Collins, A., Joseph, D. & Bielaczyc, K. (2004). Design research: Theoretical and methodological Issues. Journal of the Learning Sciences, 13(1), 15-42. http://dx.doi.org/10.1207/s15327809jls1301_2 Crosling, G. & Heagney, M. (2009). Improving student retention in higher education: Improving teaching and learning. Australian Universities Review, 51(2), 9-18. http://www.nteu.org.au/library/view/id/502 910 Australasian Journal of Educational Technology, 2012, 28(5) Freeman, M., Clarkeburn, H., & Treleaven, L. (2007). A collaborative approach to improving academic honesty. In A. Brew & J. Sachs (Eds.), Transforming a university: The scholarship of teaching and learning in practice (pp. 153-162). Sydney: Sydney University Press. http://hdl.handle.net/2123/2127 Garrison, D. R. & Anderson, T. (2003). E-learning in the 21st century: A framework for research and practice. London: RoutledgeFalmer. IMS Global Learning Consortium Inc. (2010). IMS GLC Student Induction to E-Learning: A best practices framework for promoting post secondary student retention associated with induction to e- learning. http://www.imsglobal.org/siel/index.html Kennedy, G. E., Judd, T. S., Churchward, A., Gray, K. & Krause, K. L. (2008). First year students' experiences with technology: Are they really digital natives? Australasian Journal of Educational Technology, 24(1), 108-122. http://www.ascilite.org.au/ajet/ajet24/kennedy.html Kehrwald, B., Rawlins, P. & Simpson, M. (2011). Learner experiences of online learning in a blended learning situation: Different cohorts, different needs. In Changing demands, changing directions. Proceedings ascilite Hobart 2011. (pp.671-676). http://www.ascilite.org.au/ conferences/hobart11/downloads/papers/Kehrwald-concise.pdf Ko, S. & Rossen, S. (2010). Teaching online: A practical guide (3rd ed.). New York: Routledge. Krause, K.-L. & McEwen, C. (2009). Student induction to e-learning: A progress report. University of Southern Queensland report by Link Affiliates Team within the Australian Digital Futures Institute. http://www.linkaffiliates.net.au/Publications/SIeL_Mar09_Report.html Lever, T., Mahony, M. J. & Wozniak, H. (2007). GetReal: Building and managing essential academic learning from the academic periphery. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/lever.pdf Levy, P. (2006). 'Living' theory: A pedagogical framework for the process support in networked learning. Research in Learning Technology, 14(3), 225-240. http://dx.doi.org/10.1080/09687760600837025 Motteram, G. & Forrester, G. (2005). Becoming an online distance learner: What can be learned from students’ experiences of induction to distance programmes? Distance Education, 26(3), 281-298. http://dx.doi.org/10.1080/01587910500291330 O'Donnell, C. M., Sloan, D. J. & Mulholland, C. W. (2006), Evaluation of an online student induction and support package for online learners. European Journal of Open, Distance and E- learning, 01.04.2006, 1-12. http://www.eurodl.org/?p=archives&year=2006&halfyear=1&article=220 Price, L., Richardson, J. T. E. & Jelfs, A. (2007). Face-to-face versus online tutoring support in distance education. Studies in Higher Education, 32(1), 1-20. http://dx.doi.org/10.1080/03075070601004366 Reeves, T., Herrington, J. & Oliver, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16(2), 96-115. http://dx.doi.org/10.1007/BF02961476 Reeves, T., McKenney, S. & Herrington, J. (2011). Publishing and perishing: The critical importance of educational design research. Australasian Journal of Educational Technology, 27(1), 55-65. http://www.ascilite.org.au/ajet/ajet27/reeves.html Scagnoli, N. I. (2001). Student orientations for online programs. Journal of Research on Technology in Education, 34(1), 19-27. Salmon, G. (1998). Student induction and study preparation online. Paper presented at the Telematics in Education Seminar. http://atimod.com/research/presentations/Finland2.doc Salmon, G. (2000). E-moderating: The key to teaching and learning online. London: Kogan Page. Wozniak, Pizzica and Mahony 911 Shin, N. (2003). Transactional presence as a critical predictor of success in distance learning. Distance Education, 24(1), 69-86. http://dx.doi.org/10.1080/01587910303048 Sun, P. C., Tsai, R. J., Finger, G., Chen, Y. Y. & Yeh, D. (2008). What drives a successful e- learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers & Education, 50(4), 1183-1202. http://dx.doi.org/10.1016/j.compedu.2006.11.007 Tomei, L., Hagel, H., Rineer, A., Mastandrea, L. A. & Scolon, J. (2009). Do orientation materials help students successfully complete online courses? International Journal of Information and Communication Technology Education, 5(2), 73-89. http://www.igi-global.com/article/ orientation-materials-help-students-successfully/2375 Wozniak, H. (2007). Empowering learners to interact effectively in asynchronous discussion activities. In M. Bullen & D. Janes (Eds.), Making the transition to e-learning: Strategies and issues (pp. 208-228). Hershey, PA: Idea Group Publishing. Wozniak, H., Mahony, M. J., Lever, T. & Pizzica, J. (2009). Stepping through the orientation looking glass: A staged approach for postgraduate students. Australasian Journal of Educational Technology, 25(2), 221-234. http://www.ascilite.org.au/ajet/ajet25/wozniak.html Wozniak, H., Mahony, M. J., Pizzica, J. & Koulias, M. (2007). How do students ‘get learning’? Unexpectedly diverse pathways in an activity based, online orientation site. In ICT: Providing choices for learners and learning. Proceedings ascilite Singapore 2007. http://www.ascilite.org.au/conferences/singapore07/procs/wozniak.pdf Wozniak, H. & Silveira, S. (2008). Transforming learning: Using structured online discussions to engage learners. In A. Brew & J. Sachs (Eds.), Transforming a university: The scholarship of teaching and learning in practice (pp. 163-174). Sydney: Sydney University Press. http://hdl.handle.net/2123/2128 Wozniak, H., Uys, P. & Mahony, M. J. (2012). Digital communication in a networked world. In J. Higgs, R. Ajjawi, L. McAllister, F. Trede & S. Loftus (Eds; 3rd ed), Communicating in the Health Sciences. Melbourne: Oxford University Press. Zaritsky, R., Kelly, A. E., Flowers, W., Rogers, E. & O’Neill, P. (2003). Clinical design sciences: A view from sister design efforts. Educational Researcher, 32(1), 32-34. http://dx.doi.org/10.3102/0013189X032001032 Authors: Helen Wozniak, Senior Lecturer, eLearning Coordinator Northern Territory Medical Program, FlindersNT, Flinders University PO Box 362, Charles Darwin University NT 0815 Email: Helen.Wozniak@flinders.edu.au Web: http://www.flinders.edu.au/people/helen.wozniak Jenny Pizzica, Lecturer, Institute for Interactive Media and Learning University of Technology, Sydney PO Box 123 Broadway NSW 2007 Email: Jenny.Pizzica@uts.edu.au Web: http://datasearch.uts.edu.au/iml/staff/details.cfm?StaffId=3848 Dr Mary Jane Mahony, Honorary Senior Lecturer CoCo, Faculty of Education and Social Work, University of Sydney Email: maryjane.mahony@gmail.com Web: http://sydney.edu.au/education_social_work/coco/ Please cite as: Wozniak, H., Pizzica, J. & Mahony, M. J. (2012). Design-based research principles for student orientation to online study: Capturing the lessons learnt. Australasian Journal of Educational Technology, 28(5), 896-911. http://www.ascilite.org.au/ajet/ajet28/wozniak.html