Australasian Journal of Educational Technology, 2015, 31(5). 597 Conjecture mapping to optimize the educational design research process Helen Wozniak Flinders University While educational design research promotes closer links between practice and theory, reporting its outcomes from iterations across multiple contexts is often constrained by the volumes of data generated, and the context bound nature of the research outcomes. Reports tend to focus on a single iteration of implementation without further research to determine the generalisability of the outcomes to new contexts. This paper outlines a retrospective analytical approach used to capture the significant design features of an online orientation resource implemented in one university context, and then adapted for further exploration, design and evaluation at a second contrasting university. The educational problem studied was how to support health science students to overcome barriers of transitioning to online distance study. A conjecture mapping process aligned to the three phases of educational design research provided a framework to review the outcomes at the first university. This guided the research at the second university, including choice of the data sources during implementation. This process enabled the researcher to undertake a comparative analysis and evaluate the extent of generalisability of the resource to the second university, leading to refined design principles and a framework that illustrates the student transition process. Introduction Educational design research (EDR) blends design, research and practice concurrently (Gravemeijer & Cobb, 2006; Gunn & Steel, 2012; Laurillard, 2012; McKenney & Reeves, 2014). It is a long-term approach, wherein practitioners and researchers collaborate to design a solution to a complex educational problem, which is evaluated through multiple iterations of implementation. EDR is a new and evolving methodological framework that is becoming popular within the academy (Anderson & Shattuck, 2012). It can however, give rise to complex research designs involving multiple data sources. It has been criticised because of its lack of structure and open stance towards data collection during the EDR process (Dede, 2004). This can also impact the reasoning process applied to the analysis of the large data sets that emerge from studying multiple iterations (Kelly, 2004; Reimann, 2011). One way to overcome these issues is to consider the relationship between the elements of the designed solution and map how they are expected to work together in practice, using the analysis of such conjectures to drive forward changes to further iterations or new contexts (Reimann, 2011; Sandoval, 2004, 2014). Capturing these processes can provide an interpretive framework which guides a retrospective analysis of all iterations of EDR (Reimann, 2011; McKenney & Reeves, 2014). Educational design researchers have highlighted the importance of this integrative analysis which captures the events that shape the design trajectory and explain the complexity of this process (Barab, Baek, Schatz, Scheckler, & Moore, 2008; Cobb & Gravemeijer, 2008). Analysis conducted in this way may potentially lead to the development of theoretically grounded design principles (DPs) that illustrate the knowledge generated from the conduct of EDR (Design-Based Research Collective, 2003; McKenney & Reeves, 2012; Reeves, Herrington, & Oliver, 2005). Whilst generation of DPs may contribute to the body of knowledge and development of related theories, much of the EDR conducted to date has described “small-scale interventions”, where it is “unclear if the results achieved are meeting the challenge of promoting widespread adoption of the tested interventions” (Anderson & Shattuck, 2012; p. 24). Despite this, those experienced in conducting EDR, suggest that its customisability leads to more flexible authentic research that can make an impact on educational practices, policy and theory (Fishman, Penuel, Allen, Cheng, & Sabelli, 2013; Herrington, 2012; McKenney & Reeves, 2014; Reimann, 2013). The tension between providing outcomes that have a local impact, yet progress theoretical knowledge for more distant experiences, provides another challenge for the conduct of EDR (Barab & Squire, 2004). Whilst aspiring to widespread adoption may appear to be the optimal outcome of EDR, an alternative viewpoint is suggested by Sandoval (2014). He argues that small scale EDR carried out across multiple contexts can highlight key variations, and that this may not Australasian Journal of Educational Technology, 2015, 31(5). 598 necessarily require moving from small scale to larger scale research projects. This approach towards conducting EDR across additional contexts is also noted by Plomp (2013; p. 34): Design principles and local (instruction) theories will be additionally powerful if they have been validated in the successful design of more similar interventions in various contexts. Chances for such knowledge growth will increase when design research is conducted in the framework of research programs, because then projects can build upon one another. This paper provides an example of how these two challenges (i.e., integrative data analysis and application of outcomes to new contexts) were managed. An extended EDR project studying student orientation to online distance learning was conducted sequentially, at two different universities. First, the educational problem and the theoretical basis for designing the orientation resources are described, along with the context of the research. Next, Sandoval’s (2004, 2014) conjecture mapping framework was integrated with McKenney and Reeves’ (2012) three-stage model of EDR. This framework was then used to describe the EDR conducted. The focus of this report is on how it guided the iterative reasoning process and provided a structure for retrospective analysis and comparison between the two contexts. This ensured that the outcomes of this EDR built a richer picture of what was important, and what variations might matter, when designing solutions for student transition to online distance learning at university. The educational problem and research context: Transition to online distance learning This research was located in the context of Australian health professional education where students were increasingly adopting online and distance learning as their first university experience, or returning to study at university in this new mode of learning, while balancing other work and family responsibilities (Duffy & Smith, 2013). Retention of such cohorts is reportedly low (Coates & Ransom, 2011; Rovai & Downey, 2010), so the educational problem addressed in the EDR project was how to support this transition and enable these students to manage their study in such learning environments. The problem emerged in the Faculty of Health Sciences at the University of Sydney in 2006 as a result of expansion of distance courses into the online setting; and following difficulties encountered by an increasingly diverse student cohort (mature-age, managing study, work and family). A project team consisting of academic staff and educational designers from the central university support group, was formed to develop an orientation resource to support the transition of these students into studying by distance, online modes. Although technology is becoming a ubiquitous part of everyday life, research had identified that students may not necessarily know how to apply this to mediate their learning (Creanor, Trinder, Gowan, & Howells, 2006; Kirkwood & Price, 2005). Distance educators had recognised the importance of supporting students soon after enrolment and early in their course (Motteram & Forrester, 2005; Price, Richardson, & Jelfs, 2007; Simpson 2004). Suggestions for providing support for beginning online distance learners were common, but examples mostly gave descriptive accounts of local initiatives without a strong grounding in learning theory (Bozarth, Chapman, & LaMonica, 2004; Scagnoli, 2001). This did not provide evidence-based conclusions that could be applied to other contexts. At the commencement of the research, conceptualisation of how to support online distance learners had emerged from earlier work. Activities to develop online learning skills and assist students to build knowledge through asynchronous discussions had led to improved knowledge construction and academic performance (Wozniak, 2007). Also the work of Levy (2006), identifying processes required for successful student engagement in networked learning environments, influenced the development of the initial draft design principles. Transitioning to the students’ online distance learning space was conceptualised as a period of reflective and interpersonal realignment traversing the enrolment threshold and mediated by technology (Wozniak, Mahony, Lever, & Pizzica, 2009). This concept drew from the theories of situated learning (Lave & Wenger, 1991) and social constructivism (Vygotsky, 1978). Online distance learning was conceived as an interactive process between individuals and their environment, where learning is “situated within the practices of communities and the outcomes of learning depend on the ability of the learner to participate in those practices” (Mayes & De Freitas, 2004; p. 9). The role of orientation was to assist learners to become aware of the context of online distance learning, and through guidance, scaffolding and practice, develop their online identity so that they could participate in the online learning community. The orientation resource was considered a “border object anchored in Australasian Journal of Educational Technology, 2015, 31(5). 599 different practice worlds of academic and non-academic life” (Lever, Mahony, & Wozniak, 2007; p. 572). It highlighted the need to address issues around technical requirements for online study, guidance and advice about managing and organising study time, understanding how to navigate learning resources in the university’s LMS, and to begin to traverse the pathway from home and work life to study life (Forrester, Motteram, Parkinson, & Slaouti, 2005). This concept of the transition space provided a basic framework for starting to think about how the complex combination of physical, social and technical conditions of student orientation might be addressed along an extended timeframe, from enrolment through to the early weeks of the semester. This background, combined with analysis of the context of the first university and the educational design expertise of the design team, led to developing the additional DPs outlined in Table 1. These drew on the first and second stage of Salmon’s (2000) model of online learning (access and socialisation) , the Community of Inquiry model (Garrison, Anderson, & Archer, 1999) and Moore’s (1993) theory of transactional distance. The orientation resource aimed to provide structure and dialogue, encouraging students to gain exposure to online learning skills, and share aspects of their world in a risk-free supportive environment. Table 1 Original design principles for the research Design Principle Description DP1 Orientation involves the interplay of three dimensions; technical, interpersonal and reflective practice, that blend in the transition from student’s current world (often involving work and family life) to an academic world of study. DP2 Orientation is an extended process continuing after enrolment. DP3 Orientation is aided by participation in activities within a safe risk-free environment. DP4 Interactions between content, teachers and other students is a core attribute of online learning which requires an appreciation of how knowledge is constructed, and how to communicate and collaborate with peers in mainly text-based systems. The EDR was undertaken sequentially at two university locations spanning the period from 2006 to 2011 as shown in Figure 1. In Context 1 health science students studying fully online postgraduate programs participated in three iterations of Getting-On-Track. This included an initial trial with 20 students and staff, then two further iterations; 179 students across 6 programs and 2 faculties in Semester 1 2007, and then another with 292 students from 7 programs across 3 faculties in Semester 2 2007. Lessons learnt from the EDR undertaken in the first context were then used to guide the development of a second resource at Charles Darwin University (CDU) known as Get Learning Online @ CDU (Context 2). This contrasting regional university offered an online undergraduate nursing qualification to a diverse group of students, many of whom were not able to access more traditional on-campus study modes. The resource was piloted with experts and a group of students in 2010 (first iteration) before being implemented with a large cohort of students (n = 541) in semester 1, 2011 (second iteration). Context 1 University of Sydney 2006 -2008 Context 2 Charles Darwin University 2009 -2011 1. Pilot 2006 n=20 Leading to3. Sem 2 2007 n=292 2. Sem 1 2007 n=179 1. Pilot 2010 n=50 2. Sem 1 2011 n=541 Iterations Iterations Figure 1. Iterations of EDR in the two contexts studied Australasian Journal of Educational Technology, 2015, 31(5). 600 Framework to guide the conduct of EDR Guidelines on how to implement EDR are currently evolving (Ma & Harmon, 2009). Reeves’ early model of EDR (Reeves, 2000) has been modified to highlight three phases that outline the conduct of EDR as shown in Figure 2 (McKenney & Reeves, 2012). During the first phase of EDR the problem (in this case student transition to online distance learning) was identified, and analysed through a review of the literature and exploration of the context where the designed solution was to be developed. This theoretical background informed the development of a set of draft DPs which guided the second phase of EDR, the design and construction of a practical solution to the problem being researched (an online orientation resource housed in the university’s learning management system [LMS] to assist students in making the transition to online distance learning). In the third phase, the resource was implemented through several iterations of data collection and analysis to examine how the designed solution worked in practice. The focus of the implementation was on evaluating how students accessed and engaged with the resources and associated activities. The results from iterations informed the practical and theoretical outputs of EDR; refinement of the orientation resource and DPs, and improved theoretical understanding about the transition process. As noted in the introduction, one of the criticisms of EDR as a methodology is that it lacks a clear framework for capturing the developments in each phase, and guiding the reasoning processes about the emergent theories that result from enactment of the designed solution (Kelly, 2004). Conjecture mapping has been suggested as a way of capturing the design trajectory to conceptualise reasoning that emerges from the EDR process (Cobb & Gravemeijer, 2008; Reimann, 2011; Sandoval, 2014). Phase 1 Analysis & Exploration Investigation of context and analysis of literature and learning theory informs draft design principles Phase 2 Design & Construction Design principles guide development of the resource for the specific context Phase 3 Evaluation & Reflection Cyclical iterations of implementation with analysis leading to modification of the designed solution and refinement of design principles Maturing Intervention Theoretical understanding Implementation and Spread Figure 2. Three phases of conducting EDR (Source: McKenney & Reeves, 2012; p.77) Conjecture mapping was originally conceived by Sandoval (2004; p. 214) as “embodied conjectures”. He later described conjecture mapping as a “means of specifying theoretically salient features of a learning environment design and mapping out how they are predicted to work together to produce desired outcomes” (Sandoval, 2014; p. 19). Figure 3 extends the three phases espoused by McKenney and Reeves (2012), shown previously in Figure 2, to include this conjecture mapping process utilising most of the terminology described by Sandoval (2014). Using a conjecture mapping approach enabled a clearer articulation of exactly how the draft DPs developed in Phase 1 were reified into the design of the solution in Phase 2. As will be illustrated later in Figure 4, this included identifying design elements such as the tools and materials, and what the learner and teacher was expected to do; the task and participant structures and discursive practices (Sandoval, 2014). These design elements are described by Sandoval as embodiments. In this paper, however, the more specific term design elements, is used to describe these tools, structures, and practices; the term embodiment may lead to confusion because it evokes a variety of meanings in post-structural research paradigms examining online learning (Land, 2004). Prior to embarking on iterations of implementation in Phase 3, it is advisable to first identify how it is expected that the designed solution will play out by “delineating an envisioned learning trajectory” (Reimann, 2011; p. 41). Sandoval (2014) phrases this as mediating processes, or the type of activity that is anticipated from learners engaging with the designed solution. This leads to identifying the intended outcomes that are expected to emerge from implementing the designed solution. From this information the design and theoretical conjectures can be identified prior to Phase 3. As defined by Sandoval (2014; p. 24) design conjectures are expressed as follows: “if learners engage in this activity (task + participant) structure with these tools through this discursive practice, then this mediating process will emerge”; and for theoretical conjectures “if this mediating process occurs it will lead to this outcome”. Australasian Journal of Educational Technology, 2015, 31(5). 601 Analyse orientation needs for online distance learners and develop draft design principles (DP) DPs reified into the design elements Mapping of how learners are expected to engage with the orientation resource (mediating processes) leading to the intended outcomes Guides Iterations of implementation which determines effectiveness of mapping and conjectures Selection of data sources used to evaluate Design principles for orientationModifications made to Theoretical understanding about transition to online distance learning Retrospective Analysis Further adjustments & refinements based on enactment of design Design and theoretical conjectures envisioned Phase 1 Analysis & Exploration Phase 2 Design & Construction Phase 3 Evaluation & Reflection Analysis leads to EDR Outcomes Figure 3. Aligning the conjecture mapping process to the three phases of EDR The value in this approach is that it helps the researcher to review their research questions, make choices about data sources that can be used to evaluate the implementation of the design during Phase 3, and recognise unexpected outcomes during the research. This can then help to identify new areas of analysis and exploration (Phase 1), provide a clear rationale for making changes to the designed solution (Phase 2), or guide further evaluation directions during the next iteration. Creating a conjecture map can also focus later retrospective analyses to improve the trustworthiness of the outcomes of EDR, namely the DPs and emergent theories. The application of this framework to the EDR project undertaken at the two universities will be described next. Examples of how it supported the reasoning process applied to the research are illustrated. Although it is best to construct a conjecture map prior to Phase 3 of EDR, it should be noted that since this research was conducted from 2006-2011, the conjecture maps illustrated in Figures 4 and 5 are a retrospective reconstruction using Sandoval’s (2014) more recent terminology. This original research did, however, draw on Sandoval’s (2004) earlier work to guide the EDR process undertaken. The value in applying conjecture mapping to this research was its role in identifying the data sources, and managing the large data sets during analysis of the design and theoretical conjectures. This kept the research on track, leading to refinement of the DPs, and later guided a retrospective comparative analysis across both contexts. This ultimately led to enhancement of our understanding about transition to online distance learning. Application of EDR and conjecture mapping to the research In order to demonstrate application of the EDR process depicted in Figure 3 this section will outline how the conjecture mapping process helped to inform and guide the EDR process mirroring the three phases of EDR. The overarching research question was: What guiding principles underpin the design of orientation resources that support the transition of online distance learners who are studying health professional programs at university? Moving from DPs to a conjecture map In order to represent the DPs identified from the first exploration and analysis phase of EDR outlined in Table 1; project teams needed to identify the design elements and construct the conjecture map (Sandoval, 2014; p. 19). In Context 1 the tools and materials for the orientation resource were primarily associated with the LMS and included textual content and multimedia files representing the technical aspects of transition. Six modules were designed spanning transition elements identified from Phase 1. Topics included navigating the LMS and time management, online communication, collaborating in online groups, completing assessment tasks and academic integrity. The task structures were online orientation activities (12 in total), that aimed to provide a non-threatening environment for new online learners to try out navigation, online communication, collaboration and assignment submission. The participant structures referred to how the students, and the project team members moderating their participation, were expected to take part in each of the activities. These encompassed all three dimensions of DP1, technical, interpersonal and to a lesser extent reflective aspects. The design team hoped that Australasian Journal of Educational Technology, 2015, 31(5). 602 students would complete the activities, collaborate with peers and reflect on their personal learning journey. For example, the second module, Communicating with Others, introduced Salmon’s five-stage model of online learning (Salmon, 2000), and asked students to use the email tool and engage in the discussion board. This included replying to others and joining in a group discussion to highlight the collaborative aspects of asynchronous communication (discursive practices). It was planned that moderation of the online discussions would be limited to introductory messages and some weaving and summarising of postings. Other feedback was automated (e.g., navigation quiz). The left hand section of Figure 4 identifies the relationship between the DPs from Table 1 and the design elements of the orientation resource (see also Wozniak et al., 2009). Intended Outcomes Design Elements Mediating Processes Observable Interactions - Students access content and engage with activities - Students access predominantly early in semester - Limited moderation is needed to encourage access and activity completion Participant Artefacts - Students successfully complete activities - Postings in discussion forums & collaborative activities demonstrate sharing of ideas and building of knowledge Theoretical Conjectures Design Conjectures Tools and Materials - Modules within LMS to address key challenges for online study (DP1,2,3) - LMS tools e.g., communication, assessment, collaboration (DP3,4) Task structures - Activities embedded in module content (DP3) Participant structures - Self assessment of readiness (DP1,2,3) - Student-to-student communication (DP1,4) - Feedback on successful engagement (DP3,4) - Minimal moderation (DP4) PHASE 1 and PHASE 2 conjecture map developed PHASE 3 conjecture map evaluated - Content areas in resource noted as relevant by students for development as online distance learner which is confirmed by LMS tracking - Online resource requires low levels of maintenance - Students value trialling of practice activities in safe environment (e.g. posting and replying to discussions, quiz completion, submission of assignment, use of calendar for time management) - Students report lessening of anxiety from engaging in resource and practice activities - Online discussion contributions demonstrate online collaboration and peer support Discursive practices - Sharing ideas in asynchronous discussions (DP4) Figure 4. Conjecture map for supporting student transition to online distance learning in Context 1 The next component of the conjecture map, mediating processes, are usually determined by making observations in classrooms, but in the context of this research, where students were studying online and at a distance to the university, this included other observable interactions (Sandoval, 2014). It was expected that we would observe students accessing all content areas and actively engaging with the activities, especially early in the semester. Data footprints from student access to the orientation resource (LMS tracking data), and project team notes about the level of moderation guidance provided, constituted the data sources to identify the mediating processes. Additional evidence was determined by evaluating the participant artefacts created from engagement with the orientation activities (e.g., quiz completions, content analysis of discussion postings). The last component, intended outcomes, was what we expected when students engaged with the resource, and was mostly related to confirming the draft DPs. For example, it was expected that the three dimensions of DP1 would be reflected in the content of the student survey results, indicating the relevance of each module, and value in trying out online tools to aid their transition. It was expected that the postings in the discussion activities would enact Salmon’s model with students sharing ideas and encouraging each other (technical and interpersonal elements DP1). In addition, it was also expected that these postings would reveal examples of self-reflective statements related to managing online distance learning (reflection dimension). The middle and right columns of Figure 4 illustrate these aspects. Further details about the resource design and analysis of the student engagement levels are available in other publications (Wozniak, Mahony, & Pizzica, 2012; Wozniak et al., 2009). Examples related to the evaluation of this conjecture map from enactment of EDR Phase 3 will be provided next. Australasian Journal of Educational Technology, 2015, 31(5). 603 Formulating and evaluating the design and theoretical conjectures Evaluating the implementation of the orientation resource in Phase 3 requires a clear articulation of the design and theoretical conjectures that imply linkages between the three components of the conjecture map (indicated by the arrows in Figure 4). The dark arrows in Figure 4 relate to the design and theoretical conjectures that are described in this manuscript, with the dashed arrow referring to an unexpected result noted during Phase 3. It is noteworthy that the conjecture mapping process contributed to the selection of data sources to evaluate the third phase of EDR; and also aided in identifying divergent results, such as when the expected mediating processes did not occur, or unexpected outcomes emerged. This directed changes to the earlier phases of EDR, and also fed forward to evaluation in further iterations. In Context 1 the project team and academic staff were particularly interested in researching the access patterns and pathways taken by students initially accessing the orientation resource during the second iteration. It was anticipated that students would access the resource as soon as it was available, and that the modular design would provide a pathway for students to access the practice activities within the context of the module, so that they not only participated, but understood the purpose of the activities for their development as an online distance learner. The related design conjectures were that the design elements of the orientation resource would: • Lead to accessing the orientation resource prior to and early in the semester; and, • Enable new online learners to navigate through the modules and locate and participate in the embedded activities. The project team closely monitored the LMS log files tracking individual student pathways into and through the resource from the time it was made available (7 days prior to the commencement of semester for iteration 2), and during the first 3 weeks of semester. An observable interaction noted early in Phase 3 was that some students navigated directly to the orientation activities, missing the context of the activities, and their role in development of online learning skills (Wozniak, Mahony, Pizzica, & Koulias, 2007). This unexpected result was not anticipated in the original design, and highlighted the importance of providing clear signposts for students new to the online learning environment, which led to modifications being made to DP3. This observation resulted in a change to the participant structures (dashed arrow in Figure 4). During the second iteration, the moderators instigated a series of one-to-one guidance emails to assist students who appeared to be missing essential information, while exploring the orientation resource. This led to improved completion of activities, and greater engagement in the discussion activities, rather than lurking and not actively participating. Prior to the commencement of the third iteration, the design elements were modified to improve the navigational signposts linking orientation resource content to the related activity; but resourcing issues meant that a lower level of monitoring and support was provided. Analysis confirmed that fewer students completed activities when less moderation support was provided (Wozniak et al., 2012). This led to further theorising about the role of moderation and one-on-one support for students who are new to online learning, highlighting the importance of teacher presence to encourage students to actively participate (DP4). Log files also indicated that access peaked the earlier the resource was available, with students who began the online resource earlier being more likely to participate in the activities. Consequently, when the resource was made available earlier in the third iteration (14 days prior to semester), active engagement before the semester started, was confirmed (DP2). It was also noted in all iterations that students continued to access the orientation resource throughout the semester suggesting an extended transition period (DP2). The research undertaken in Context 1 also aimed to determine if participation in the orientation resource aided student transition, by overcoming technical barriers, and promoting the importance of interpersonal elements such as collaboration with their peers (DP1). Therefore one theoretical conjecture (noted by the dark lines between the mediating processes and intended outcomes in Figure 4) was: • If the students engaged in the orientation resource as designed, they would report the relevance of the content included in the resource and note the value of engaging in a risk-free environment to their development as an online distance learner. This was evaluated primarily through a short survey conducted at the end of each module. Results gained by analysing the survey results and asynchronous discussion postings, highlighted the importance of trying out technologies associated with online distance learning in a safe environment (e.g., navigating Australasian Journal of Educational Technology, 2015, 31(5). 604 within an LMS, participating in a discussion activity and trying out assignment submission), and also the interpersonal aspects such as the value of sharing ideas with their peers (DP1). The LMS data and survey results also illuminated a much more extended transition journey continuing throughout the semester. Space does not allow a full explanation of all the aspects of this conjecture map so readers can refer to other publications related to this EDR project (Wozniak et al., 2012; Wozniak et al., 2007). Table 2 describes the key findings from this research with the left columns showing the relationship between the DPs summarised from Table 1, and the main research results from Context 1. For completeness, the right hand columns illustrate further refinements to the DPs following the conduct EDR at the second university. Components of the EDR undertaken at Context 2 will be outlined next. Table 2 Key findings from the EDR conducted at two university contexts with resultant modifications to the design principles (DPs) Summary of key elements from original DPs (from Table 1) Results from Context 1 (Postgraduate, higher level of moderation) Results from Context 2 (Undergraduate, lower level of moderation, addition of demographic data and student interviews Modified DP (following EDR conducted at two university contexts and a retrospective comparative analysis) DP1 Three dimensions of transition - technical, interpersonal and reflective Technical and interpersonal dimensions were evident. Technical and interpersonal dimensions evident; transition was multilayered and enhanced by personal attributes such as resilience and self-efficacy. DP1 Orientation entails a broad transition space encompassing many elements beyond managing the LMS for learning. It is primarily focused on the interplay of technical and interpersonal dimensions, that blend in the transition from the student’s current world (often involving work and family life), to an academic world of study aided by self- regulatory mechanisms. DP2 Extended process following enrolment Highest access prior to semester, extending throughout the semester. Highest access associated with early enrolment, extending throughout semester. Access linked to specific demographics (over 25 years, first online distance course). DP2a Orientation is an extended process, early enrolment and access increases participation, which is less, but still evident throughout the semester. DP2b Student engagement with orientation resources will vary. Orientation should target learners studying their first online distance course and older age groups. DP3 Participation in activities Diverse access pathways, One-on-one support assisted navigation and increased engagement with activities, Feedback highly valued. Links between content and activities can be enhanced through quest-based challenge diagrams and cartoons to stimulate interest, Automated feedback should be provided if possible. DP3a Orientation is aided by an opportunity to practice using online learning skills which stimulate curiosity and embed automated feedback on student performance. DP3b Practice activities should be clearly situated in the context of online learning with clear navigation prompts and supports that acknowledge non-linear pathways used by students to explore resources. Australasian Journal of Educational Technology, 2015, 31(5). 605 DP4 Interaction to promote collaboration in asynchronous discussions Teacher presence assists in moving lurkers to more active participation, which encourages collaboration in the text-based activities. Students highly valued audio and video recordings that mimic teacher presence. Students seek out their own networks using social media platforms. DP4 Interactions between content, teachers and other students is a core attribute of online learning, which requires an appreciation of how knowledge is constructed, and how to communicate and collaborate with peers using a range of online systems, both text and audio based, including non-university based environments. Orientation to LMS is only one element of transition. DP5 Transition is a broad concept that extends beyond orientation to the online learning environment, which also includes choices of study load, enhancement of academic and information literacy skills. Applying the lessons learnt to a second context The iterative nature of EDR led to the application of the EDR outcomes from Context 1 to a different context at Charles Darwin University, where a similar educational problem and orientation challenge was identified in 2009. During the first analysis and exploration phase at this second university, the DPs were adapted to fit the undergraduate context, and incorporate recent literature. For example, diverse access patterns were noted at the first university, and literature from the period between studying these two contexts, confirmed that students use technology in very diverse ways (Bennett, Maton, & Kervin, 2008; Bullen, Morgan, & Qayyum, 2011; Jelfs & Richardson, 2012; Kennedy, Judd, Churchward, Gray, & Krause, 2008). Consequently, new directions for researching the student transition experience were identified and enriched the EDR process. The project team was able to examine the relationship between student demographics (e.g., age, gender, full or part-time study, first online course, date enrolled, academic results) and access to the online orientation resource, which was not able to be studied in Context 1. When designing the orientation resource at the second university particular attention was paid to improving the navigation of the orientation resource, which was once again housed within the LMS. Explanations and signposts such as start here, along with pictorial representations and audio recordings, described the relationship between each of the modules and activities within the resource, as well as their relevance to beginning online distance learners. The orientation resource consisted of eight sections, with the activities organised as a quest, with cartoons to prompt interest. Content covered areas such as getting started as an online distance learner, communication, collaboration and assessment. Due to the large classes and low resources, teacher presence was created through audio and video supports. Feedback about activity participation was automated wherever possible, with minimal moderation of the discussion activities. Although contrary to the experience from Context 1 and DP4, this was a pragmatic response to the resourcing issues at the second university. The conjecture map from these modifications is illustrated in Figure 5 where the design elements are aligned to each DP. The intended outcomes were enriched to include aspects that would add to the understanding from recent literature about student transition. This was aided by the additional data sources available in the third evaluation phase (e.g., student demographics and also a small number of in depth interviews with students). Australasian Journal of Educational Technology, 2015, 31(5). 606 Intended OutcomesDesign Elements Mediating Processes Observable Interactions - Students access content and engage with activities - Students access as soon as available - Limited moderation needed to encourage access and activity completion Participant Artefacts - Students successfully complete activities - Postings in discussion forums demonstrate sharing of ideas and building of knowledge Theoretical Conjectures Design Conjectures Tools and Materials - LMS content designed to promote curiosity (cartoons) with clear navigation signposts (DP1,2,3) - LMS tools e.g., communication, assessment, collaboration, inclusion of synchronous classroom and plagiarism activity (DP3,4) - Available prior to semester (DP2) - Audio and videos of students and teachers describing survival skills (DP4) Task structures - Quest-like focus for activities (DP3) Participant structures - Self assessment of readiness (DP1,2,3) - Student-to-student communication (DP1,4) - Automated feedback on successful engagement (DP3,4) - Minimal moderation (DP4) - Content areas in resource noted as relevant by students for development as online distance learner which is confirmed by LMS tracking & interviews - Extent of access varies according to student characteristics - Access leads to retention and successful academic outcomes - Online resource requires low levels of maintenance - Students value trialling of practice activities in safe environment - Students’ experience of online distance learning illustrates awareness of key constructs (e.g,. persistence, time management) and skill development that was included in the resource - Online discussion contributions demonstrates online collaboration Discursive practices - Sharing ideas in asynchronous discussions (DP4) Figure 5. Conjecture map capturing the EDR project in Context 2. Italics are used to highlight differences between the conjecture map at Context 1 and Context 2 Drawing on the experiences at the first university and the additional data sources available, the design conjecture related to the timing of access was modified as follows: The design elements of the orientation resource would: • Lead to access soon after enrolment, prior to, and early in the semester, with some students returning to explore throughout the semester, just-in-time. The project team also considered that mediating processes associated with access and engagement might be influenced by factors such as age, mode of study, prior online study experiences, time of enrolment and assessment results; so formulated a theoretical conjecture to consider this avenue of investigation: • Student engagement with the orientation resource (i.e., access and completion of activities) may vary according to their demographic characteristics; may be related to retention and persistence with study, and correlate with assessment results. Wozniak (2013) reported the results of Phase 3 where the LMS data was matched to the student characteristics to evaluate outcomes of two iterations in Context 2. Results indicated that early enrolment was associated with greater engagement with the resource content and activities (DP2a). There was also a statistically significant association between accessing the resource, viewing pages within the resource and engaging with the activities for the population of students who were over 25 years of age, studying by distance mode, and their first online course at the university (DP2b). There was however no association between engagement and academic achievement or retention. Evidence for wide diversity in engagement with the resource reinforced the notion of targeting supports to discrete groups of students, and raised further questions about the relationship between learner dispositions, persistence and achievement in online distance study modes. The right-hand columns of Table 2 summarise the results from Context 2. This research has highlighted the importance of planning an EDR project, including how conjecture mapping assisted in all the phases of the EDR process, and directed the associated data analysis. Researchers such as Sandoval (2013), also recognised the importance of context, and its central role in influencing the design elements, mediating processes and intended outcomes, which may be unanticipated. In this EDR, conjecture mapping helped the researchers to recognise when features of the design contributed to unexpected outcomes, and enabled identification of deviations from the expected Australasian Journal of Educational Technology, 2015, 31(5). 607 design trajectory. This led to further analysis and investigation, not in the original research plan. Comparisons between outcomes from multiple iterations and different contexts can help to develop more generalised theories regarding the educational problem being studied. The next section provides an overview of this retrospective comparison, illustrating how this assisted in managing data overload, and promoting transferability of the research outcomes. Comparative analysis and discussion EDR which reports the implementation of a designed solution in different contexts, with different characteristics, needs to carefully consider its claims regarding creation of generalisable research outcomes (Kelly, 2006). Using conjecture mapping to direct the comparative analysis of two contrasting contexts, one involving undergraduate students and the other postgraduate; enabled this EDR to generate both practical and theoretical outcomes. From a practical sense, the key design features expressed in the form of DPs, can be used by other practitioners to develop supports for online distance learners in their context. Analysis of the outcomes from both contexts also demonstrated analytic generalisation (Yin, 2009); where research outcomes can lead to development of a broader theory. McKenney and Reeves (2014; p.138) applied analytic generalisation to EDR; suggesting that when “designs are tested in multiple settings and under varying conditions, or when design features are systematically varied under similar conditions, theory development can occur.” For this research, engaging in such a comparative analysis, led to improved theoretical understanding about the student transition process for online distance learners. These practical and theoretical outcomes will be described next. Results from both contexts indicated that students adopted a wide variety of access patterns when engaging in orientation resources, and that the earlier the resources are made available, the greater the participation (see Table 2, DP2a, 3b). Overall, the postgraduate students from Context 1, demonstrated greater access and participation with activities; although in Context 2 the undergraduate population over 25, studying their first online course, almost reached a similar level of engagement (DP2b). Students who did engage in the resources found the content relevant, and valued the opportunity to try out using the tools required by online distance learners (DP3a & b). It was noted in Context 2, that students appreciated the teacher presence created from audio rather than text-based mediums, and that students will utilise other non-university based tools such as social media to develop supportive learning communities (DP4). These refined DPs are illustrated in Table 2 (right-hand column) and provide guidelines, not certainties, for educational design researchers to apply to other contexts (Plomp, 2013). The EDR also aimed to evaluate the original three theoretical dimensions (technical, interpersonal and reflective) underpinning the design (DP1), and determine their relevance to student transition to online distance learning. In Context 1, survey data and comments made by students during the discussion activities, were analysed to inform the validity of these dimensions. This was expanded to include student interviews in Context 2. Consequently, a range of barriers and enablers associated with student transition to online distance learning were identified, indicating that students primarily focused on the technological and interpersonal dimensions of their transition. Results also highlighted further aspects which were not originally anticipated, such that students perceive transition as encompassing a broad array of orientation needs. These included selecting a manageable study load, mastering academic writing and appraising information sources. This led to the creation of a new DP (Table 2, DP5). This also drove the reformulation of DP1 (see Table 2). The role of large LMS datasets in educational research is an avenue that is only just beginning to be addressed by the data mining and learning analytics research community (Clarke & Nelson, 2013; Macfadyen, Dawson, Pardo, & Gasevic, 2014). Of particular interest for future research is how to harness this data to improve the design of learning experiences (Lockyer, Heathcote, & Dawson, 2013). Comparisons made between the two contexts led to conceptualising a transition theory, which posits that transition for online distance learners is a multi-layered concept similar to “transition as becoming” described by Gale & Parker (2014; p. 735). Students needed to adjust to becoming a university student by mastering academic writing, (a concept more prevalent in the second undergraduate setting), and time- management; yet they expected that university should fit around their complex lives. They also needed to make adjustments to multiple online environments which were far broader than the LMS, including accessing and evaluating information, as well as planning their study pathways through to completion of their program. A further layer of transition was associated with bridging the transactional distance Australasian Journal of Educational Technology, 2015, 31(5). 608 (Moore, 1993), which was both a physical and a psychological distance. This was lessened by scaffolding activities within the orientation resource; although it was noted that students initiated their own self- directed supports through social media, to connect with their peers. Strong intrinsic motivational factors aided in a further layer of transition associated with students developing their professional identity as a health professional. It was also hypothesized that their transition process was aided by self-regulatory behaviours, which enabled some students to better manage their individual situation, persist with study, and achieve their goals (Wozniak & McEldowney, 2015). There is emerging evidence of the importance of self-regulation for online distance study (Cho & Shen, 2013). It was concluded that universities may need to target different layers of this transition; offer diverse, flexible just-in-time supports, which should be varied according to the characteristics of the students; and provide these over a longer period of time, extending beyond the traditional pre-semester orientation week. In both contexts where the research was undertaken, the orientation resources in their designed format did not continue to be utilised for student orientation. Continuation was impacted by institutional changes to the LMS, and staffing changes in the e-learning support teams, both of which have been found to influence the likelihood of initiatives continuing (McGill, Klobas, & Renzi, 2014). This does highlight that the spread of design solutions can be impacted by forces at play in authentic settings. The value of EDR to the educational community may lie in its ability to create theoretically-grounded, empirically- tested DPs, which can be adopted and modified by others. In the context of this exemplar, the reporting of the DPs in peer-reviewed publications and informal presentations, did lead to the development of a new orientation resource in Context 2. This new resource was designed by staff not involved in the original research, with features consistent with the DPs generated from this research. This resource was also made available in 2014 to a wider student group, encompassing all learners using the LMS at the university. The pathway to greater spread of the outcomes from EDR is an area that requires further investigation. One avenue that may provide a richer insight is closer examination of the macro, meso and micro institutional layers impacting educational designs, and their interconnections. Readers could refer to works by Zawacki-Richter and Anderson (2014) or Ellis and Goodyear (2010) for further discussion of these themes. Conclusion Adopting a conjecture mapping framework aligned to the three phases of EDR strengthens the iterative nature of this approach. It provides a planning and evaluation framework, articulates the design and theoretical conjectures, and reveals unexpected findings. It also ensures that the research is focused on key features of the design. If educational design researchers develop a conjecture map in the first two phases of EDR and then use it to evaluate implementation of the designed solution, data collection is focused on capturing the design trajectory as it occurs in authentic practice environments. This is then available for retrospective analyses, allowing the researcher to trace these results back to the design, and evaluate the linkages between each component of the conjecture map. Studying student orientation to online distance learning at two universities also demonstrated that EDR does not necessarily require implementation and spread to larger scaled adoption. The two different contexts provided opportunities to review the student transition phenomena from diverse angles. A key outcome of EDR from this research was not the designed solution (orientation resource), rather the refinements that were made to the DPs and enhancement to our theoretical understanding about transitioning to online distance learning. Acknowledgements This research would not have been possible without the support of a large group of dedicated educational designers and academic staff at both universities, committed to enhancing university supports for new online distance learners. The author is particularly indebted to Dr Mary Jane Mahony, Dr Tim Lever and the USyd eLearning Initiative staff as well as Dr Liz Devonshire and Elizabeth Rosser who worked closely with academic staff on the project at CDU. Thanks also to Dr Robyn Philip and the reviewers who provided helpful critique during the preparation of this manuscript. This research also formed part of my doctoral study at CDU under the supervision of Professor Rose McEldowney (CDU) and Professor Shane Dawson (University of South Australia). Australasian Journal of Educational Technology, 2015, 31(5). 609 References Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research. Educational Researcher, 41(1), 16-25. doi:10.3102/0013189X11428813 Barab, S. A., Baek, E.-O., Schatz, S., Scheckler, R., & Moore, J. (2008). Illuminating the braids of change in a web-supported community. In A. E. Kelly, R. A. Lesh, & J. Y. Baek (Eds.), Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching (pp. 320-352). New York, NY Routledge. Barab, S. A., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1-14. doi:10.1207/s15327809jls1301_1 Bennett, S., Maton, K., & Kervin, L. (2008). The "Digital Natives" debate: A critical review of the evidence. British Journal of Educational Technology, 39(5), 775-786. doi:10.1111/j.1467- 8535.2007.00793.x Bozarth, J., Chapman, D. D., & LaMonica, L. (2004). Preparing for distance learning: Designing an online student orientation course. Educational Technology & Society, 7(1), 87-106. Retrieved from http://www.ifets.info/journals/7_1/10.pdf Bullen, M., Morgan, T., & Qayyum, A. (2011). Digital learners in higher education: Generation is not the issue. Canadian Journal of Learning and Technology, 37(1). Retrieved from http://www.cjlt.ca/index.php/cjlt/article/view/550 Cho, M.-H., & Shen, D. (2013). Self-regulation in online learning. Distance Education, 34(3), 290-301. doi:10.1080/01587919.2013.835770 Clarke, J., & Nelson, K. (2013). Perspectives on learning analytics: Issues and challenges. Observations from Shane Dawson and Phil Long. The International Journal of the First Year in Higher Education, 4(1), 1-8. doi:10.5204/intjfyhe.v4i1.166 Coates, H., & Ransom, L. (2011). Dropout DNA, and the genetics of effective support. AUSSE Research Briefings v.11 (11th ed.). Camberwell: Australian Council for Educational Research. Retrieved from http://research.acer.edu.au/ausse/1/ Cobb, P., & Gravemeijer, K. (2008). Experimenting to support and understand learning processes. In A. E. Kelly, R. A. Lesh, & Y. J. Baek (Eds.), Handbook of design research methods in education (pp. 68- 95). New York, NY Routledge. Creanor, L., Trinder, K., Gowan, D., & Howells, C. (2006). LEX: The learner experience of e-learning – Final project report. Glasgow Caledonian University, Glasgow. Retrieved from http://www.jisc.ac.uk/whatwedo/programmes/elearningpedagogy/learneroutcomes.aspx Dede, C. (2004). If design-based research is the answer, what is the question? A commentary on Collins, Joseph, and Bielaczyc; diSessa and Cobb; and Fishman, Marx, Blumenthal, Krajcik, and Soloway in the JLS Special Issue on Design-Based Research. Journal of the Learning Sciences, 13(1), 105-114. doi:10.1207/s15327809jls1301_5 Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. doi:10.3102/0013189x032001005 Duffy, E., & Smith, M. (2013). Major current themes in health professional education. In S. Loftus, T. Gerzina, J. Higgs, M. Smith, & E. Duffy (Eds.), Educating health professionals: Becoming a university teacher (pp. 313-322). Rotterdam: SensePublishers. Ellis, R. A., & Goodyear, P. (2010). Students' experiences of e-learning in higher education: The ecology of sustainable innovation. New York, NY: Routledge. Fishman, B. J., Penuel, W. R., Allen, A.-R., Cheng, B. H., & Sabelli, N. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. National Society for the Study of Education, 112(2), 136-156. Retrieved from https://nsse- chicago.org/Free/26_3092.pdf Forrester, G., Motteram, G., Parkinson, G., & Slaouti, D. (2005). Going the distance: students' experiences of induction to distance learning in higher education. Journal of Further & Higher Education, 29(4), 293-306. doi:10.1080/03098770500353185 Gale, T., & Parker, S. (2014). Navigating change: A typology of student transition in higher education. Studies in Higher Education, 39(5), 734-753. doi:10.1080/03075079.2012.721351 Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. doi:10.1016/S1096-7516(00)00016-6 http://www.ifets.info/journals/7_1/10.pdf http://www.cjlt.ca/index.php/cjlt/article/view/550 http://research.acer.edu.au/ausse/1/ http://www.jisc.ac.uk/whatwedo/programmes/elearningpedagogy/learneroutcomes.aspx https://nsse-chicago.org/Free/26_3092.pdf https://nsse-chicago.org/Free/26_3092.pdf Australasian Journal of Educational Technology, 2015, 31(5). 610 Gravemeijer, K., & Cobb, P. (2006). Design research from a learning design perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 17-51). London: Routledge. Gunn, C., & Steel, C. (2012). Linking theory to practice in learning technology research. Research in Learning Technology, 20(2), 1-14. doi:10.3402/rlt.v20i0.16148 Herrington, J. (2012). Design-based research: Implementation issues in emerging scholar research. Paper presented at the World Conference on Educational Multimedia, Hypermedia and Telecommunications 2012, Denver, Colorado, USA. Retrieved from http://www.editlib.org/p/40719 Jelfs, A., & Richardson, J. T. E. (2012). The use of digital technologies across the adult life span in distance education. British Journal of Educational Technology, 44(2), 338-351. doi:10.1111/j.1467- 8535.2012.01308.x Kelly, A. E. (2004). Design research in education: Yes, but is it methodological? Journal of the Learning Sciences, 13(1), 115-128. doi:10.1207/s15327809jls1301_6 Kelly, A. E. (2006). Quality criteria for design research: Evidence and commitments. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (pp. 107-118). London: Routledge. Kennedy, G. E., Judd, T. S., Churchward, A., Gray, K., & Krause, K. L. (2008). First year students' experiences with technology: Are they really digital natives? Australasian Journal of Educational Technology, 24(1), 108-122. Retrieved from http://ascilite.org.au/ajet/submission/index.php/AJET/article/view/1233 Kirkwood, A., & Price, L. (2005). Learners and learning in the twenty-first century: What do we know about students' attitudes towards and experiences of information and communication technologies that will help us design courses? Studies in Higher Education, 30(3), 257-274. doi:10.1080/03075070500095689 Land, R. (2004). Issues of embodiment and risk in online learning. In R. Atkinson, C. McBeath, D. Jonas- Dwyer, & R. Phillips (Eds.), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference, (pp. 530-538). Perth. Retrieved from http://www.ascilite.org/conferences/perth04/procs/land.html Laurillard, D. (2012). Teaching as a design science: Building pedagogical patterns for leaning and technology. New York, NY: Routledge. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press. Lever, T., Mahony, M. J., & Wozniak, H. (2007). GetReal: Building and managing essential academic learning from the academic periphery. In Proceedings of ICT: Providing choices for learners and learning conference. (pp. 570-577). Singapore. Retrieved from http://www.ascilite.org.au/conferences/singapore07/procs/lever.pdf Levy, P. (2006). 'Living' theory: A pedagogical framework for the process support in networked learning. ALT-J, Research in Learning Technology, 14(3), 225-240. doi:10.1080/09687760600837025 Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439-1459. doi:10.1177/0002764213479367 Ma, Y., & Harmon, S. W. (2009). A case study of design-based research for creating a vision prototype of a technology-based innovative learning environment. Journal of Interactive Learning Research, 20(1), 75-93. Retrieved from http://www.editlib.org/p/25226/ Macfadyen, L. P., Dawson, S., Pardo, A., & Gasevic, D. (2014). Embracing big data in complex educational systems: The learning analytics imperative and the policy challenge. Research & Practice in Assessment, 9(2), 17-28. Retrieved from http://www.rpajournal.com/dev/wp- content/uploads/2014/10/A2.pdf Mayes, T., & De Freitas, S. (2004). JISC e-learning models desk study: Stage 2 review of e-learning theories, frameworks and models. e-Learning Models Study: JISC. Retrieved from http://www.webarchive.org.uk/wayback/archive/20081225020952/http://www.jisc.ac.uk/uploaded_do cuments/Stage%202%20Learning%20Models%20%28Version%201%29.pdf McGill, T. J., Klobas, J. E., & Renzi, S. (2014). Critical success factors for the continuation of e-learning initiatives. The Internet and Higher Education, 22(July), 24-36. doi:10.1016/j.iheduc.2014.04.001 McKenney, S., & Reeves, T. C. (2012). Conducting educational design research. Oxon: Routledge. McKenney, S., & Reeves, T. C. (2014). Educational design research. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 131-140). New York, NY: Springer. http://www.editlib.org/p/40719 http://ascilite.org.au/ajet/submission/index.php/AJET/article/view/1233 http://www.ascilite.org/conferences/perth04/procs/land.html http://www.ascilite.org.au/conferences/singapore07/procs/lever.pdf http://www.editlib.org/p/25226/ http://www.rpajournal.com/dev/wp-content/uploads/2014/10/A2.pdf http://www.rpajournal.com/dev/wp-content/uploads/2014/10/A2.pdf http://www.webarchive.org.uk/wayback/archive/20081225020952/http:/www.jisc.ac.uk/uploaded_documents/Stage%202%20Learning%20Models%20%28Version%201%29.pdf http://www.webarchive.org.uk/wayback/archive/20081225020952/http:/www.jisc.ac.uk/uploaded_documents/Stage%202%20Learning%20Models%20%28Version%201%29.pdf Australasian Journal of Educational Technology, 2015, 31(5). 611 Moore, M. G. (1993). Theory of transactional distance. In D. Keegan (Ed.), Theoretical principles of distance education (pp. 22-38). London: Routledge. Motteram, G., & Forrester, G. (2005). Becoming an online distance learner: What can be learned from students’ experiences of induction to distance programmes? Distance Education, 26(3), 281-298. doi:10.1080/01587910500291330 Plomp, T. (2013). Educational design research: An introduction. In T. Plomp, & N. Nieveen (Eds.), Educational design research Part A: An introduction (pp. 11-50). Enschede: SLO Netherlands Institute for Curriculum Development. Retrieved from http://international.slo.nl/publications/edr/ Price, L., Richardson, J. T. E., & Jelfs, A. (2007). Face-to-face versus online tutoring support in distance education. Studies in Higher Education, 32(1), 1-20. doi:10.1080/03075070601004366 Reeves, T. (2000). Enhancing the worth of instructional technology research through ‘design experiments’ and other development research strategies. Paper presented at the Annual Meeting of the American Educational Research Association New Orleans, LA. Retrieved from http://www.teknologipendidikan.net/wp-content/uploads/2009/07/Enhancing-the-Worth-of- Instructional-Technology-Research-through3.pdf Reeves, T., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16(2), 96-115. doi:10.1007/bf02961476 Reimann, P. (2011). Design-based research. In L. Markauskaite, P. Freebody & J. Irwin (Eds.), Methodological choice and design: Scholarship, policy and practice in social and educational research (pp. 37-50). Dordrecht: Springer Science Business Media. Reimann, P. (2013). Design-based research - designing as research. In R. Luckin, S. Puntambekar, P. Goodyear, B. Grabowski, J. Underwood & N. Winters (Eds.), Handbook of design in educational technology (pp. 44-52). New York, NY: Routledge. Rovai, A. P., & Downey, J. R. (2010). Why some distance education programs fail while others succeed in a global environment. The Internet and Higher Education, 13(3), 141-147. doi:10.1016/j.iheduc.2009.07.001 Salmon, G. (2000). E-moderating: The key to teaching and learning online. London: Kogan Page. Sandoval, W. A. (2004). Developing learning theory by refining conjectures embodied in educational designs. Educational Psychologist, 39(4), 213-223. doi:10.1207/s15326985ep3904_3 Sandoval, W. A. (2013). Educational design ressarch in the 21st centtury. In R. Luckin, S. Puntambekar, P. Goodyear, B. Grabowski, J. Underwood, & N. Winters (Eds.), Handbook of design in educational technology (pp. 388-396). New York, NY: Routledge. Sandoval, W. A. (2014). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23(1), 18-36. doi:10.1080/10508406.2013.778204 Scagnoli, N. I. (2001). Student orientations for online programs. Journal of Research on Technology in Education, 34(1), 19-27. doi:10.1080/15391523.2001.10782330 Simpson , O. (2004). The impact on retention of interventions to support distance learning students. Open Learning: The Journal of Open, Distance and e-Learning, 19(1), 79-95. doi:10.1080/0268051042000177863 Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (M. Cole, V. John-Steiner, S. Scribner & E. Souberman, Trans.). Cambridge, MA: Harvard University Press. Wozniak, H. (2007). Empowering learners to interact effectively in asynchronous discussion activities. In M. Bullen, & D. Janes (Eds.), Making the transition to e-learning: Strategies and issues (pp. 208- 228). Hershey, PA: Idea Group Publishing. Wozniak, H. (2013). Student Engagement with an online orientation resource: How learning analytics refines educational design principles. In H. Jan, C. Alec, & I. Valerie (Eds.), World Conference on Educational Multimedia, Hypermedia and Telecommunications 2013 (pp. 941-950). Victoria, Canada: AACE. Retrieved from http://www.editlib.org/p/112074/ Wozniak, H., Mahony, M. J., Lever, T., & Pizzica, J. (2009). Stepping through the orientation looking glass: a staged approach for postgraduate students. Australasian Journal of Educational Technology, 25(2), 221-234. Retrieved from http://ajet.org.au/index.php/AJET/article/view/1151 Wozniak, H., Mahony, M. J., & Pizzica, J. (2012). Design-based research principles for student orientation to online study: Capturing the lessons learnt. Australasian Journal of Educational Technology, 28(5), 896-911. Retrieved from http://ajet.org.au/index.php/AJET/article/view/823 Wozniak, H., Mahony, M. J., Pizzica, J., & Koulias, M. (2007). How do students ‘get learning’? Unexpectedly diverse pathways in an activity based, online orientation site. In Proceedings of ICT: http://international.slo.nl/publications/edr/ http://www.teknologipendidikan.net/wp-content/uploads/2009/07/Enhancing-the-Worth-of-Instructional-Technology-Research-through3.pdf http://www.teknologipendidikan.net/wp-content/uploads/2009/07/Enhancing-the-Worth-of-Instructional-Technology-Research-through3.pdf http://www.editlib.org/p/112074/ http://ajet.org.au/index.php/AJET/article/view/1151 http://ajet.org.au/index.php/AJET/article/view/823 Australasian Journal of Educational Technology, 2015, 31(5). 612 Providing choices for learners and learning conference. (pp. 1104-1108). Singapore: Retrieved from http://www.ascilite.org.au/conferences/singapore07/procs/wozniak.pdf Wozniak, H., & McEldowney, R. (2015). Layers of transition: the lived experiences of online distance learners. In T. Thomas, E. Levin, P. Dawson, K. Fraser, & R. Hadgraft (Eds.), Research and development in higher education: Learning for life and work in a complex world (38, pp. 505-515). Melbourne, Australia. Retrieved from http://www.herdsa.org.au/wp- content/uploads/conference/2015/HERDSA_2015_Wozniak.pdf Yin, R. K. (2009). Case study rresearch: Design and methods (4th ed.). Thousand Oaks, CA: SAGE Inc. Zawacki-Richter, O., & Anderson, T. (2014). Online distance education: Towards a research agenda. Edmonton: AU Press, Athabasca University. doi:10.15215/aupress/9781927356623.01 Retreived from http://www.aupress.ca/index.php/books/120233 Corresponding author: Helen Wozniak, helen.wozniak@flinders.edu.au Australasian Journal of Educational Technology © 2015. Please cite as: Wozniak, H. (2015). Conjecture mapping to optimize the educational design research process. Australasian Journal of Educational Technology, 31(5), 597-612. http://www.ascilite.org.au/conferences/singapore07/procs/wozniak.pdf http://www.herdsa.org.au/wp-content/uploads/conference/2015/HERDSA_2015_Wozniak.pdf http://www.herdsa.org.au/wp-content/uploads/conference/2015/HERDSA_2015_Wozniak.pdf http://www.aupress.ca/index.php/books/120233 mailto:helen.wozniak@flinders.edu.au