Microsoft Word - 247-2549-6-CE.doc Australasian Journal of Educational Technology, 2013, 29(5). ascilite 699 Enhancing an instructional design model for virtual reality- based learning Chwen Jen Chen and Chee Siong Teh Universiti Malaysia Sarawak, Malaysia In order to effectively utilize the capabilities of virtual reality (VR) in supporting the desired learning outcomes, careful consideration in the design of instruction for VR learning is crucial. In line with this concern, previous work proposed an instructional design model that prescribes instructional methods to guide the design of VR-based learning environments. This article provides a thorough elaboration on how formative research is employed to enhance the earlier model. The study has successfully generated five new hypothesized principles to enhance the robustness of the instructional design model through the formative research process. The newly derived hypothesized principles also provide insights into the design of various experimental studies for testing them in the effort to form a more comprehensive guide for the design of VR-based learning environments. Introduction Virtual reality systems can be implemented on affordable personal computers using conventional mouse and keyboard input devices. These systems have lower costs and the potential to be more widely used (Ausburn & Ausburn, 2004; Chuah, Chen & Teh, 2011; Jenson & Forsyth, 2012). Their teaching utility depends on how applications are designed to assist learning (Bakas & Mikropoulos, 2003; Barrett & Blackledge, 2012; Dodd & Antonenko, 2012). Although general instructional design models and theories are useful, more specific ones are needed to exploit the affordances a particular information technology provides (Reigeluth & Frick 1999). Given that it is now possible to display three dimensional graphics and interact with them in real time to replicate real world environments virtually, the question remains as to when providing these environments is needed and how the environments and the ways of interacting with them can be structured to maximize effectiveness. There are various VR systems ranging from high fidelity, immersive systems to lower fidelity non- immersive ones. This article elaborates an instructional design model originally proposed by Chen, Toh, and Wan (2004), and based on systematic applications of the model to develop and formatively evaluate simulations on a non-immersive VR system. Figure 1 illustrates the model. Figure 1. The instructional design model proposed by Chen, Toh, and Wan (2004). Generally, the model combines the concept of integrative goals (Gagné & Merrill, 1990) with a model for designing constructivist learning environments (Jonassen, 1999). They serve as the macro-strategy, which according to Reigeluth and Merrill (1978), concerns the selection, sequence, and organization of the subject-matter to be presented. Table 1 provides a summary of the principles employed in the macro- strategy. Additionally, a number of design principles, derived from the cognitive theory of multimedia Australasian Journal of Educational Technology, 2013, 29(5). 700 learning (Mayer, 2002) as well as the research findings reported in Chen and Wan (2008), serve as the micro-strategy that basically, involves how to effectively present the content. Table 2 provides a brief explanation for each of these principles. Table 1 Principles of the macro-strategy Principle Description Objectives Identifying the types of learning (labels, verbal information, intellectual skills and/or cognitive strategies) and the respective learning objectives (Gagné & Merrill, 1990) Integrative goals Determining the integrative goals by combining several interrelated objectives that are to be integrated into a comprehensive purposeful activity, which is called an enterprise (Gagné & Merrill, 1990) Enterprise scenario/problem Identifying the enterprise scenario that must be played out in conducting the enterprise. (Gagné & Merrill, 1990). It is similar to the problem posed in a constructivist learning environment by Jonassen (1999, 2002). This problem comprises three integrated components: problem context, problem representation, and problem manipulation space (Jonassen, 1999). Support tools Providing various interpretative and intellectual systems to support constructivist learning through the problem posed. These may include related cases, information resources, and various cognitive tools (Jonassen, 1999). Instructional activities Providing instructional activities to support constructivist learning, which include modelling, coaching, and scaffolding (Jonassen, 1999) Table 2 Principles of the micro-strategy Principle Description Spatial contiguity Learners learn better when corresponding words and pictures are presented near rather than far from each other on the screen (Mayer, 2002) Coherence Learners learn better when extraneous words, pictures, and sounds are excluded rather than included (Mayer, 2002). Modality Learners learn better from animation and narration than from animation and on-screen text (Mayer, 2002). Redundancy Learners learn better from animation and narration than from animation, narration, and on-screen text (Mayer, 2002). Guided exploration Learners learn better when visual navigation aid is provided as it enables them to stay oriented while navigating in a virtual environment (Chen & Wan, 2008). Related work Studies have been conducted to examine the design issues of VR-based learning environments from various perspectives. A few years after Chen, Toh, and Wan's (2004) model was introduced, Nelson and Erlandson (2008) have proposed some design solutions for an educational multi-user virtual environments (MUVE) that are in line with the multimedia learning principles (Mayer, 2005) and cognitive load theory (Mayer and Moreno, 2003). However, these design solutions are yet to be verified through empirical studies. At the more micro level, Nelson (2007) has found out that the individualized, reflective guidance system which is embedded within a MUVE-based scientific inquiry curriculum has improved the learning of the targeted learners. The study has proposed some design guidelines for the development of such a guidance system. Incorporating visual cues in desktop virtual environments has also been found to be helpful in Australasian Journal of Educational Technology, 2013, 29(5). 701 reducing the feeling of disorientation and to aid navigational tasks (Dickey, 2004; Smith & Marsh, 2004). Roussou, Oliver, and Slater (2006) have investigated the effect of interactivity on conceptual learning in an immersive virtual environment and the results suggest that the passive virtual environment in which activity was guided by a virtual robot produces better conceptual change than the fully interactive virtual environment. The survey conducted by Huang, Rauch, and Liaw (2010) on their web-based three-dimensional learning environment concludes that providing highly interactive learning experiences is essential in such learning environments. It also points out the appropriateness of such tools to promote creativity in problem solving, serve as a scaffolding tool and increase motivation. However, Huang et al. (2010) do not provide any guidelines on how to specifically manipulate the unique characteristics of of VR to promote learning. Formative research This article focuses on enhancing the instructional design model using formative research. The work entails what many have called design and development research. Design and development research is a relatively new approach in studying educational design and development activities (Richey & Klein, 2007). Van den Akker (1999) uses the term "development research" to refer to various kinds of research approaches used in design and development work, such as design-based research, formative research, and action research. Reeves, Herrington, and Oliver (2004) equate development research to design research. Richey and Klein (2007), on the other hand, regard design-based research and formative research as two innovative approaches within the scope of design and development research. Consensus is still lacking on a consistent term to define this emerging paradigm of educational inquiry. In design-based research, the aim is to refine a particular educational intervention to optimize its success as well as to derive a more general mechanism that underlies it (The Design-Based Research Collective, 2003). Design-based researchers systematically adjust various aspects of the designed intervention and test each in context to generate new theories, artefacts, and practices that affect learning (Barab & Squire, 2004). Reigeluth and Frick (1999) suggest the formative research approach to produce new design theories/models as well as to enhance the existing ones. Formative research, which is qualitative in nature, has been used to improve existing instructional design theories and models, such as the elaboration theory (English & Reigeluth, 1996; Kim, 1994), a theory to facilitate understanding (Roma, 1990), a theory for the design of computer-based simulations (Shon, 1996), and a theory for designing instruction for teams (Armstrong, 1993). In addition, Lee and Reigeluth (2003) use formative research to improve the heuristic task analysis process. Focus of the study The VR model (Figure 1) was used as an initial structure that was formatively evaluated in this study to produce a more robust design model. To do this, it was necessary to identify methods that are part of the model as either working or otherwise. Hence, the specific objectives of this study are to: a) Refine an instructional design model for VR-based learning environments using the formative research methodology. b) Hypothesize an improved instructional design model for VR-based learning environments based on the outcomes of the formative research. The research question of this study is: What aspects of the instructional design model can be potentially improved? To answer this research question, it is necessary to identify design methods that are working and those that are not working so well. This may eventually lead to changes in the existing methods, inclusion of new methods, exclusion of existing methods, a more detailed guide on how to accomplish particular parts of the method, and/or possible variations in the methods for different situations, such as for learners with different learning styles or learning environments for different contexts. Australasian Journal of Educational Technology, 2013, 29(5). 702 Method – formative research The underlying logic of formative research is that, if an accurate application of an instructional design model is created, then any weaknesses that are found in the application may reflect weaknesses in the model, and any enhancements identified for the application may reflect ways to enhance the model, at least for some subset of the situations for which the model is intended (Reigeluth & Frick, 1999). Formative procedure The process for conducting the formative research in this study was adapted from the process suggested by Reigeluth and Frick (1999) and involved: a) Selecting a design theory (or model). As mentioned earlier, this study chose the instructional design model for VR-based learning environments as proposed by Chen et al. (2004). b) Designing an instance of the model, which is a specific application of the design model. This study involved an expert of the model to ensure the design instance was as pure an instance of the design model as possible to avoid two types of weaknesses; omission and commission. c) Conducting a pilot study. d) Collecting and analysing formative data on the instance. The intent is to identify and remove problems in the instance, explore consequences of adding new elements to or removing existing elements from the design instance as well as to reconfirm the appropriateness of methods prescribed by the model. e) Varying six elements of the model to create multiple instances of its application and evaluating them formatively. f) Repeating data collection and analysis cycles to confirm earlier findings. g) Offering tentative revisions for the model. The six elements of the model that were systematically varied included navigation speed, environmental richness, collision detection, coaching message, feedback type and input device. Design instance The design instance was a VR-based learning environment for novice car drivers to learn about traffic rules and traffic signs. It consisted of five three-dimensional virtual road scenarios that allowed the learner to have real-time exploration and interaction. The instructional design model (Chen et al., 2004) guided its design and Chen et al. (2004) also provides a thorough description on how the learning environment was designed based on the macro and micro strategies. Elements of the design instance The navigation speed, environmental richness, collision detection, coaching message, feedback type, and input device used are among the major elements that could be attuned in a virtual environment to optimize its effectiveness as a learning environment. The settings of these elements on the design instance were varied for evaluation to determine which settings were perceived by the participants as user-friendly and/or helpful and did not interfere with the learning process. The elements and their respective settings are shown in Table 3. Table 3 Elements and their respective settings Variable Settings Navigation speed fast (25fps), medium (18fps), slow (10fps) Environmental richness rich, bare Collision detection Coaching message on, off available, unavailable Feedback type text, narration, text and narration Input device mouse, keyboard Australasian Journal of Educational Technology, 2013, 29(5). 703 Navigation speed Navigating through the virtual environment in real-time is a distinct feature of VR, but which speeds should be allowed, which are preferred, and which will not inhibit the learning process? Three designs were created with fast, medium and slow speeds and all other features held constant to determine whether speed differences were noticeable and, if so, which speed was desirable. The fast, medium and slow speeds were set at 25, 18 and 10 frames per second (fps) respectively as such fps settings will show distinct speed differences when navigating through the virtual environment. Environmental richness This element refers to the various virtual objects that are not directly influencing the learning process but are incorporated into the environment to enhance its attractiveness and realism. According to Godden and Baddeley (1975), environmental richness could provide situational context, which refers to the surroundings in which knowledge and meaning making are present. This context could be remembered through episodic memory, along with the actual intended learning content (Tulving, 1993). On the contrary, Jones and Dumais (1986) claim there is a possibility for the environmental detail to interfere with memory of the actual learning content. Thus, two designs were created that varied this element but with other features being the same, one adding trees, buildings, bushes and so forth while another only had roads and signs. Collision detection This element refers to the ability to stop movement when users are too close to a virtual object to avoid colliding or moving through it. A virtual environment with collision detection mimics our real world. Two designs were created, one with collision detection and one without, with other features remaining constant. Coaching message The model assumes coaching enhances performance. Thus, two designs were created, one with coaching and one without, while all other features remained the same. Figure 2 shows a screenshot of the coached design with a pop-up message. This pop-up message directs a learner to a particular hyperlink for obtaining additional information that might assist him/her in solving the problems posed. Figure 2. Screenshots of the coached design instance. Feedback type Feedback is a crucial component of any learning process (Brown, 1997; Jonassen & Hannum, 1987; Laurillard, 1993; Norman & Schmidt, 1992) as it informs learners about the consequences of their actions and motivates them to further interact with a system (Vasilyeva, Pechenizkiy, & Puuronen, 2006). Information presented by feedback can be of different qualities and modalities (Mayes, 2002). As text and narration are two feasible feedback modalities to be incorporated into this non-immersive virtual environment, three feedback designs were created: one with text messages, one with narrated messages, and one with combined text and narrated messages. Hyperlinks for additional information hin Hint statement Coaching message – requesting the learner to refer to a particular hyperlink Australasian Journal of Educational Technology, 2013, 29(5). 704 Input device The study of this element involved creating two designs: one that required the use of a mouse and another one that required the use of a keyboard. These two devices are most commonly used in non-immersive VR systems and are ubiquitously available thus, making them feasible for use in learning using such simulations. Pilot study A pilot study of the one-to-one evaluation method was conducted to ensure (a) the appropriateness and adequacy of an interaction guide which contained a list of questions that the experimenter could pose to a participant to prompt a continuous and an effective think-aloud process, (b) the proper functioning of the Noldus Observer system which was used to record the sessions, (c) the appropriateness of the evaluation procedure, and (d) the clarity of the interview questions as well as debriefing questions. It also provided practice for the experimenters. It involved a number of representative participants who were not involved in the subsequent evaluation. This pilot study was conducted in the human-computer interaction laboratory used for the actual evaluation sessions. The interaction guide was found to be adequate to trigger ideas for the experimenter to ask various questions for prompting the think-aloud process, and the Noldus Observer system had successfully recorded the participants' interactions with the learning environment as well as produced the synchronized video recording of their behaviour. Some changes were made to the evaluation procedure. For example, in the pilot study, every participant was asked to explore a particular learning environment and then the experimenter would launch another learning environment that had a different setting for one of its elements for the participant's further exploration. The act of launching by the experimenter which was observed by the participant had made it obvious that the newly launched environment was probably different. Thus, a change was made to the evaluation procedure in which all learning environments with different element settings were launched beforehand and the experimenter would only be required to switch between these learning environments in the actual evaluation session. Some changes were also made to the interview questions by removing jargon and breaking long questions into shorter ones. Participants Learners include anyone who is eligible to attend the Road Transport Department novice car driver instruction program but has yet to undertake it. The study involved twelve female and nine male participants who fulfilled the above-stated criteria and were randomly selected among the first-year undergraduate students of the researcher's university. A series of three one-to-one evaluations with three different participants for each element was conducted except for the coaching message element that involved six participants. The one-to-one evaluations involved thinking aloud, interviews, and direct observation with audio-video recording. The study also involved another ten participants to provide feedback on the methods prescribed in the macro-strategy of the model. These ten participants evaluated the original learning environment in which they only experienced the original settings of the six elements. A different set of participants was employed for this macro-strategy evaluation so as to avoid confusion that may arise if participants from the one-to-one evaluations who had experienced elements with different settings were chosen. Data collection Having observation, think aloud and guided interviews during the one-to-one evaluation sessions as well as debriefing at the end allow for triangulation of data. Instruments This study involved the use of two sets of instruments: an interview question set and a debriefing question set (Appendix). Interview question set The micro-strategy questions of the interview guide progressed from very open-ended to targeted ones and focused on identifying particular aspects of the design instance implementation that helped or hindered learning and ways to improve weak elements. Open-ended questions were used to trigger each Australasian Journal of Educational Technology, 2013, 29(5). 705 participant to voice their thoughts while solving a task. This helped to reveal unforeseen factors that either helped or hindered his/her learning. Each participant was asked targeted questions that were specific to each of the six manipulated elements. The interview questions for macro-strategy asked participants specifically about the seven prescribed methods in the model's macro-strategy, including: problem context, representation, manipulation space, information resources, coaching, scaffolding, and constructivism. Debriefing question set These questions gave participants an opportunity to reflect on and evaluate the implementation of design instance as a whole, point out any strengths and weaknesses, and make any additional comments. Procedure Macro-strategy The study required every participant to explore the entire learning environment. During the exploration, the experimenter posed specific questions related to methods prescribed in the macro-strategy to the participants. The participants then proceeded to the debriefing session. Micro-strategy For each element in Table 3, every participant was required to explore each design with pre-determined settings. For example, in investigating navigation speed, each participant used fast, then medium and finally slow speeds. If the participant was able to notice the differences and decided on his or her preferred setting, the experimenter terminated the use of the design and proceeded to the debriefing session. Otherwise, the experimenter repeated the evaluation with the subsequent virtual road scenario. A similar procedure was employed for all the other elements in Table 3. Results and discussion Macro-strategy Integrative goal The findings from this evaluation show that explicitly revealing the value and focus of a learning environment assists in engaging the learner with the learning activities. All the participants were motivated to further explore the learning environment after reading information on its importance and learning goal. As pointed out by Archer and Hughes (2011), learners achieve better if they understand the learning goals and expected outcomes. Problem context, problem manipulation space Participants perceived the learning environment as a novel and safer approach to learn such content and were excited about its game-like approach. Papastergiou (2009); Prensky (2001); Tüzün, Yılmaz-Soylu, Karakuş, İnal, and Kızılkaya (2009) are among many others who agree on the capability of game approach to raise learners' motivation and facilitate their learning in an engaging and joyful manner. Problem representation In this learning environment, the three-dimensional virtual road scenarios serve as an important representation of the problem. All the participants agreed that this representation helped them understand traffic rules and signs in the real world. Unlike a book, which was often used to study such content, this representation allowed them to experience various road scenarios and practise solving various problems without risk. Such three-dimensional representation affords learning by facilitating tasks that require spatial processing (Dalgarno & Lee, 2010; Wallet, Sauzéon, Rodrigues, and N'Kaoua, 2009; McComas, Pivik, & Laflamme, 1998). Generally, the realism of the problem representation was able to fulfil the participants' expectation. Instructional strategies (scaffolding and modelling)   All participants were able to distinguish the complexity differences of the problems posed. They agreed that the sequencing of problems from simple to complex helped in their learning. Arranging learning tasks from simple to complex arises out of cognitive learning theory and is supported by early Australasian Journal of Educational Technology, 2013, 29(5). 706 instructional design theories (see Reigeluth, 1983). According to Sweller, van Merriënboer, and Paas (1998) using highly complex learning tasks at the beginning of a learning session would yield excessive cognitive load for the learners and subsequently decreases their learning, performance and motivation. The participants also agreed that the behavioural modelling that demonstrated how virtual vehicles abide by the rules in various road scenarios in the virtual environment and the subsequent cognitive modelling that articulated the reasoning for such behaviour were useful to inform them of the traffic rules before their virtual cars encountered similar situations in the virtual environment. These findings help to confirm the appropriateness of incorporating such instructional activities into the learning environment. Support tool (information resources) Eight out of the ten participants agreed during debriefing that the few hyperlinks that were available for each scenario were obvious to them. However, a cross-checking with the video recording of the one-to- one evaluation session showed only two participants accessed these hyperlinks during their navigation through the virtual road scenarios and another two participants accessed them before starting their exploration through virtual road scenarios. Participants who accessed the hyperlinks agreed these hyperlinks contained information resources that were useful. This helps to confirm the appropriateness of this support tool. However, recordings showed participants were so engrossed in exploring the virtual world that they did not bother to refer to other sources of information available on the interface outside it, suggesting access to these support tools be incorporated into the virtual environment itself. Instructional strategy (coaching) Three participants also commented that the frequent appearance of the pop-up message was a bit irritating, particularly when the message was repeated. The learning environment sometimes provided coaching via pop-up text messages with narration when it detected the learner breaking a traffic rule. Generally, the coaching message would request the learner to refer to a related hyperlink available on the interface outside the virtual world. A similar message would be repeatedly shown by the learning environment whenever the learner broke the same type of rule. Observation via the video recording revealed that eight out of ten participants read all the messages but somehow only two of them responded to them by referring to the hyperlinks. Failing to get most participants to refer to the hyperlinks indicates that the way coaching is presented via pop-up text message with narration is somehow ineffective in getting the participants to refer to the information resources. In line with this, Bowman et al. (2003) have stated that users of a virtual environment typically prefer to examine abstract information within context in the three-dimensional world. Abstract information is defined by Bowman et al. (2003) as the information that is not directly perceptible in the physical world. Thus, the findings of this study provide insight into the possibility of embedding the information available via the hyperlink into the virtual road scenarios to ease the access to it. Such embedding would also reduce the use of pop-up messages for this purpose. Constructivism All the participants also agreed that they preferred to learn traffic rules and traffic signs using this learning environment, which allowed them to construct their knowledge while trying to solve the various problems posed in a more authentic context. Such responses further support the appropriateness of applying constructivism as the underlying pedagogical approach of this model. Very early advocates of constructivism as the appropriate theoretical foundation of virtual reality learning environments include Bricken (1990) and Winn (1993). This belief continues to be supported by researchers such as Chen and Teh (2000); Greening (1998); Jonassen, Hernandez-Serrano, and Choi (2000); and Neale, Brown, Cobb, and Wilson (1999), and it remains in recent studies such as Barrett, Blackledge, and Coyle (2011), Garcia and Pacheco (2013), Huang et al. (2010), Meggs, Greer, and Collins (2012), and so forth. Micro-strategy The feedback obtained through the guided interviews revealed consistent responses for elements on navigation speed and collision detection while responses for environmental richness, coaching message, feedback type, and input device were inconsistent. All participants of the navigation speed variable preferred normal speed and all participants of the collision detection variable were unable to notice whether or not collision detection was activated. Australasian Journal of Educational Technology, 2013, 29(5). 707 Navigation speed Recordings revealed that fast navigation in the virtual environment made it difficult to control the car. Speed of movement is regarded as a navigation problem in a virtual environment (Sayers, 2004). Fast navigation often leads to disorientation (Miller, 1994). Most participants also expressed their frustration with slow navigation. This is consistent with Barfield and Hendrix's (1995) finding which indicates that the level of presence was significantly less when using an update rate of 5 and 10 fps (slow navigation) when compared to update rates of 20 and 25 fps (fast navigation). Collision detection The effect of collision detection did not seem to be noticeable. When the collision detection was turned on, a participant would not be allowed to navigate out of a road as some transparent blocks were put up to prevent such occurrences. However, when it was turned off, a few participants were observed to navigate along the road side. This is in line with the observation by Bowman, Hodges, Allison, and Wineman (1999) that keeping a user within an environment with simple collision detection can prevent some disorientation. Environmental richness There were also inconsistencies in preferences for environmental richness. This finding is in line with Ragan, Huber, Laha, and Bowman's (2012) study which reported that such environmental detail does not affect learning outcomes. Input device In debriefings, participants raised concerns about the type of input used to navigate and the clarity of the image and text on the virtual traffic signs. Table 4 shows the sample responses for the stated debriefing questions. Many reported difficulty in controlling the mouse, indicating a need for alternative input, such as a steering wheel or joy-stick. The finding is consistent with the recent study by Garcia-Ruiz, Edwards, Aquino-Santos, Tashiro, and Kapralos (2011). The participants of their study were requested to compare the use of a gaming joy-stick, mouse, and keyboard for navigating through a non-immersive collaborative virtual environment known as Realtown. These participants reported their preference for the keyboard and reported facing difficulties in handling the mouse. Visual clarity The responses to debriefing questions also revealed that text legibility was a problem. Although visual clarity is a well-known principle in designing any learning environment, additional attention should be given when incorporating information visualization into a three-dimensional virtual environment. The level of detail and scale of the text are rendered by the system with respect to a user's scale viewing frustum and this affects the legibility of text. Bowman et al. (2003) suggest such text information should be defined as 3-D billboards which always appear orthogonal to the viewer's orientation to ensure its legibility in a virtual environment. Coaching message The effectiveness of coaching was inferred through the participant access to hyperlinks. Participants did not access these on their own when coaching was absent. Even with coaching, only one in three did. This is consistent with the macro-strategy findings as described in the previous section. Hence, these findings point to the need to further devise a more effective method to relay the coaching message. Possible ways include varying the message content, incorporating more attractive multimedia elements onto the message and/or reducing such messages by embedding related support tools into the virtual environment. Australasian Journal of Educational Technology, 2013, 29(5). 708 Table 4 Sample responses for debriefing questions Participant Response Issue raised Q: If you would change only one thing in the system, what would it be? P1 I would like to change the control device for the navigation inside the system. Input type P7 It is very hard to control the car. Input type P9 I want to change the control. I don't like using the mouse for my navigation. Input type P10 I want to change the control. I don't like using the mouse to turn left and right. Input type P19 The control Input type P21 I want to change the input device to steering. Input type P13 Make the situations more realistic. Clarity Q: What are the weaknesses of the system? P7 It is very hard to control the car. Input type P8 Hard to control the car in the system using the mouse. Input type P14 Controlling the car using the mouse. Input type P19 Hard to control the car in the system using a mouse Input type Q: Are there any other comments of the system? P7 If possible, it is better to use keyboard than to use a mouse for the controls. Input type Q: What are the weaknesses of the system? P3 I think the visualization is poor. It is not very clear and I find it quite difficult to see. It seems blurry. Clarity P4 I think the signboards should be more clear and visible. Clarity P7 I would like to change the quality of the graphics. Clarity Hypotheses to enhance the instructional design model Figure 3 illustrates the instructional design model that incorporates hypotheses generated from the interpretations of these formative research findings. As elaborated in Chen et al. (2004), the process of designing a VR-based learning environment starts from the centre of the circular shaped diagram of the macro-strategy (innermost ring) and gradually moves outward to the outmost ring. Instructional designers will need to identify individual objectives and the relationships among these objectives to derive the integrative goal. They will then need to create an enterprise scenario which involves selecting the problem context, problem representation and problem manipulation space that will help in achieving the integrated goal. The virtual environment will represent the problem, and provide a space for learner's to perform learning activities. The design process continues by providing various necessary supports that may assist the learners to actively construct their knowledge in the learning environment. These supports include related cases, information resources, cognitive tools, collaboration and conversation tools, and social or contextual support. Coaching, scaffolding and modelling are the main strategies that will be employed in instructional activities. Having completed the design at the macro level, Mayer's (2002) principles of multimedia design which serves as the micro-strategy are incorporated into the macro structure. These principles guide the design of instructional messages in the learning environment in the effort to produce better learning. In addition to the methods prescribed by the original model, this study produces five hypotheses related to the principles of the micro-strategy. Although the findings do not suggest any hypothesis for the macro- strategy, they help to confirm the appropriateness of existing methods in the macro-strategy as well as Australasian Journal of Educational Technology, 2013, 29(5). 709 provide insights into the generation of hypothesized new principles for the micro-strategy. Table 5 states and briefly describes the hypothesized principles. Figure 3. The enhanced instructional design model. Table 5 The hypothesized principles Manipulated Element Hypothesized Principle Brief description Navigation speed Appropriate navigation speed Should provide appropriate speed to enable accurate, easy and comfortable navigation through the three-dimensional virtual environments. Environmental richness Visual clarity Should incorporate clear images and legible text into virtual environments, particularly if the learning environment emphasizes those elements. Collision detection Nonessential collision detection Detecting collision is not crucial in a virtual environment that simulates real world context. Nevertheless, detecting collision could sometimes serve as an intervention to eliminate or prevent erroneous understanding while navigating and interacting with a virtual environment. Coaching message Support tools access Should incorporate the access of these support tools from within the virtual environment itself. Input device Alternative input device/interface There is a need to identify input devices or navigational metaphors that are congruent with the task to be performed. Note. The findings through the manipulation of the feedback type element were inconsistent and thus, do not lead to any hypothesized principle. Australasian Journal of Educational Technology, 2013, 29(5). 710 Conclusion The study has generated a total of five hypothesized new principles through the collection and analysis of data obtained from representative users of the learning environment. However, the applicability of these principles beyond the context used in this study has not yet been empirically tested. The design instance simulates an observable real-world context and thus, the hypothesized principles may not be applicable, for example, to a design instance that simulates an artificial context. Hence, subsequent research studies should look at different types of content, learners, and learning situations to assure generalizability of the findings (English & Reigeluth, 1996). Various experimental studies that involve larger sample sizes can also be designed to further test the hypothesized principles. Acknowledgments The author acknowledges the financial support rendered by Universiti Malaysia Sarawak through Fundamental Research Grant Scheme, Ministry of Higher Education, Malaysia. References Archer, A., & Hughes, C. A. (2011). Explicit instruction: Effective and efficient teaching. New York: The Guilford Press. Armstrong, R. B. (1993). The team instructional prescriptions (TIP) theory: A set of integrated models for prescribing instructional strategies for teams. Unpublished doctoral dissertation, Indiana University Graduate School, Bloomington, IN. Ausburn, L. J., & Ausburn, F. B. (2004). Desktop virtual reality: a powerful new technology for teaching and research in industrial teacher education. Journal of Industrial Teacher Education, 41(4), 33-58. Bakas, C., & Mikropoulos, T. (2003). Design of virtual environments for the comprehension of planetary phenomena based on students' ideas. International Journal of Science Education, 25(8), 949-967. Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1-14. Barfield, W., & Hendrix, C. (1995). The effect of update rate on the sense of presence within virtual environments. Virtual Reality, 1(1), 3-15. Barrett, M., & Blackledge, J. (2012). Evaluation of a prototype desktop virtual reality model developed to enhance electrical safety and design in the built environment. ISAST Transactions on Computing and Intelligent Systems, 3(3), 1-10. Barrett, M., Blackledge, J., & Coyle, E. (2011). Using virtual reality to enhance electrical safety and design in the built environment. ISAST Transactions on Computers and Intelligent Systems, 3(1), 1-9. Bowman, D. A., Hodges, L. F., Allison, D., & Wineman, J. (1999). The educational value of an information-rich virtual environment. Presence, 8(3), 317–331. Bowman, D. A., North, C., Chen, J., Polys, N. F., Pyla, P. S., & Yilmaz, U. (2003, October). Information- rich virtual environments: theory, tools, and research agenda. In Proceedings of the ACM symposium on virtual reality software and technology (pp. 81-90). ACM. Bricken, W. (1990). Learning in virtual reality. [Editorial] Human interface technology laboratory, University of Washington. Brown, A. (1997). Designing for learning: What are the essential features of an effective online course? Australian Journal of Educational Technology, 13(2), 115-12. Australasian Journal of Educational Technology, 2013, 29(5). 711 Chen, C. J., & Teh, C. S. (2000). An affordable virtual reality technology for constructivist learning environments. In G. K. Yeo (Ed.), Proceedings of the 4th Global Chinese Conference on Computers in Education (pp. 414-421). Singapore. Chen, C. J., Toh, S. C., & Wan, M. F. (2004). The theoretical framework for designing desktop virtual reality-based learning environments. Journal of Interactive Learning Research, 15(2), 147-167. Chen, C. J., & Wan, M. F. (2008). Guiding exploration through three-dimensional virtual environments: A cognitive load reduction approach. Journal of Interactive Learning Research, 19(4), 579-596. Chuah, K. M., Chen, C. J., & Teh, C. S. (2011). Designing a desktop virtual reality-based learning environment with emotional consideration. Research and Practice in Technology Enhanced Learning, 6(1), 25-42. Dalgarno, B., & Lee, M. J. W. (2010).What are the learning affordances of 3-D virtual environments? British Journal of Educational Technology, 41(1), 10-32. Dickey, M. D. (2004). An architectural perspective for the design of educational virtual environments. Journal of Visual Literacy, 24(1), 49-66. Dodd, B. J., & Antonenko, P. D. (2012). Use of signaling to integrate desktop virtual reality and online learning management systems. Computers & Education, 59(4), 1099-1108. English, R. E., & Reigeluth, C. M. (1996). Formative research on sequencing instruction with the elaboration theory. Educational Technology Research & Development, 44(1), 23-42. Gagné, R. M., & Merrill, M. D. (1990). Integrative goals for instructional design. Educational Technology Research and Development, 38(1), 23-30. Garcia-Ruiz, M. A., Edwards, A., Aquino-Santos, R., Tashiro, J., & Kapralos, B. (2011). Towards usable collaborative virtual environments for promoting listening comprehension. In V. Giovanni & Braman, J. (Eds.), Multi-User Virtual Environments for the Classroom: Practical Approaches to Teaching in Virtual Worlds (pp. 386-411). USA: Information Science Reference (IGI Global). Garcia, I., & Pacheco, C. (2013). A constructivist computational platform to support mathematics education in elementary school. Computers & Education, 66, 25-39. Godden, D. R., & Baddeley, A. D. (1975). Context-dependent memory in two natural environments: on land and underwater. British Journal of Psychology, 66(3), 325-331. Greening, T. (1998). Building the constructivist toolbox: An exploration of cognitive technologies. Educational Technology, 38(2), 28-35. Huang, H. M., Rauch, U., & Liaw, S. S. (2010). Investigating learners' attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers & Education, 55(3), 1171- 1182. Jenson, C. E., & Forsyth, D. M. (2012). Virtual reality simulation: Using three-dimensional technology to teach nursing students. Computer Informatics Nursing, 30(6), 312-318. Jonassen, D. H. (1999). Designing constructivist learning environments. In C. M. Reigeluth (Ed.), Instructional-design theories and models: A new paradigm of instructional theory (Vol. 2, pp. 215- 239). New Jersey: Lawrence Erlbaum Associates. Jonassen, D. H. (2002). Integration of problem solving into instructional design. In R. A. Rieser & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (pp. 107-120). New Jersey: Merrill Prentice Hall. Australasian Journal of Educational Technology, 2013, 29(5). 712 Jonassen, D. H., & Hannum, W. H. (1987). Research-based principles for designing computer software. Educational Technology, 27(12), 7-14. Jonassen, D. H., Hernandez-Serrano, J., & Choi, I. (2000). Integrating constructivism and learning technologies. In M. Spector & T. M. Anderson (Eds.), Integrated and holistic perspectives on learning, instruction and technology: Understanding complexity (pp.103-127). Netherlands: Kluwer Academic. Jones, W. P., & Dumais, S. T. (1986). The spatial metaphor for user interfaces: experimental tests of reference by location versus name. ACM Transactions on Information Systems, 4, 42-63. Kim, Y. (1994). Formative research on the simplifying conditions method for task analysis and sequencing of instructional content. (Unpublished doctoral dissertation). Indiana University Graduate School, Bloomington, IN. Laurillard, D. (1993). Rethinking university teaching: A framework for effective use of educational technology. London: Routledge. Lee, J., & Reigeluth, C. M. (2003). Formative research on the heuristic task analysis process. Educational Technology Research & Development, 51(4), 5-24. Mayer, R. E. (2002). Multimedia learning. Cambridge: Cambridge University Press. Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 31–48). New York: Cambridge University Press. Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43–52. Mayes, J.T. (2002). The technology of learning in a social world. In R. Harrison, F. Reeve, A. Hanson, J. Clarke (Eds.), Supporting lifelong learning: Perspectives on learning (Vo1. 1). Routledge: London. McComas, J., Pivik, J., & Laflamme, M. (1998). Children's transfer of spatial learning from virtual reality to real environments. CyberPsychology & Behavior, 1(2), 121-128. Meggs, S. M., Greer, A., & Collins, S. (2012). Virtual reality in interior design education: Enhanced outcomes through constructivist engagement in Second Life. International Journal of Web-Based Learning and Teaching Technologies, 7(1), 19-35. doi:10.4018/jwltt.2012010102 Miller, L. D., 1994. Metrics for usability standards in computing (MUSiC): a usability evaluation of the Rolls-Royce virtual reality for aero engine maintenance system (Unpublished master's thesis). University College, London. Neale, H. R., Brown, D. J., Cobb, S. V. G., & Wilson, J. R. (1999). Structured evaluation of virtual environments for special needs education. Presence: Teleoperators and Virtual Environments, 8(3), 264-282. Nelson, B. C. (2007). Exploring the use of individualized, reflective guidance in an educational multi-user virtual environment. Journal of Science Education and Technology, 16(1), 83-97. Nelson, B. C., & Erlandson, B. E. (2008). Managing cognitive load in educational multi-user virtual environments: reflection on design practice. Educational Technology Research and Development, 56(5), 619-641. Norman, G. A., & Schmidt, H. G. (1992). The psychological basis of problem-based learning: A review of the evidence. Academic Medicine. 67, 557–565. Australasian Journal of Educational Technology, 2013, 29(5). 713 Papastergiou, M. (2009). Digital game-based learning in high school computer science education: impact on educational effectiveness and student motivation, Computers & Education, 52(1), 1-12. Prensky, M. (2001). Digital game-based learning. New York: McGraw Hill. Ragan, E. D., Huber, K. J., Laha, B., & Bowman, D. A. (2012). The effects of navigational control and environmental detail on learning in 3D virtual environments. IEEE Virtual Reality, 1(1), 11-14. Reeves, T. C., Herrington, J., & Oliver, R. (2004). A development research agenda for online collaborative learning. Educational Technology Research and Development, 52(4), 53-65. Reigeluth, C. M. (Ed.). (1983). Instructional design theories and models. Hillsdale, NJ: Lawrence Erlbaum Associates. Reigeluth, C. M. & Frick, T. W. (1999). Formative research: A methodology for creating and improving design theories. In C. M. Reigeluth (Ed.), Instructional-design theories and models – a new paradigm of instructional theory (pp. 633-652). New Jersey: Lawrence Erlbaum. Reigeluth, C. M., & Merrill, M. D. (1978). A knowledge base for improving our methods of instruction. Educational Psychologist, 13, 54-70. Richey, R. C., & Klein, J. D. (2007). Design and development research: Methods, strategies, and issues. Mahwah, NJ: Lawrence Erlbaum. Roma, C. (1990). Formative evaluation research on an instructional theory for understanding. (Unpublished doctoral dissertation). Indiana University Graduate School, Bloomington, IN. Roussou, M., Oliver, M., & Slater, M. (2006). The virtual playground: An educational virtual reality environment for evaluating interactivity and conceptual learning. Virtual Reality, 10(3), 227-240. Sayers, H. (2004). Desktop virtual environments: a study of navigation and age. Interacting with Computers, 16(5), 939-956 Shon, M. (1996). Formative research on an instructional theory for the design of computer-based simulations for teaching causal principles. Unpublished doctoral dissertation, Indiana University Graduate School, Bloomington, IN. Smith, S. P., & Marsh, T. (2004). Evaluating design guidelines for reducing user disorientation in a desktop virtual environment. Virtual Reality, 8(1), 55-62. Sweller, J., van Merriënboer, J. J. G., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296. The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. Tulving, E. (1993). What is episodic memory? Current Directions in Psychological Science, 2, 67-70. Tüzün,H., Yılmaz-Soylu, M., Karakuş, T., İnal, Y., Kızılkaya, G. (2009). The effects of computer games on primary school students' achievement and motivation in geography learning. Computers & Education, 52(1), 68-77. van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training. The Netherlands: Kluwer. Australasian Journal of Educational Technology, 2013, 29(5). 714 Vasilyeva, E., Pechenizkiy, M., Puuronen, S. (2006). The challenge of feedback personalization to learning styles in a web-based learning system. Proceedings of the 6th IEEE International Conference on Advanced Learning Technologies. (pp. 1143-1144). IEEE Computer Society. Wallet, G., Sauzéon, H., Rodrigues, J., & N'Kaoua, B. (2009). Transfer of spatial knowledge from a virtual environment to reality: Impact of route complexity and subject's strategy on the exploration mode. Journal of Virtual Reality and Broadcasting, 6(4), 572-574. Winn, W. (1993). A conceptual basis for educational applications of virtual reality (Technical Report TR-93-9). Seattle, Washington: Human Interface Technology Laboratory, University of Washington. Retrieved from http://www.hitl.washington.edu/publications/r-93-9/ Corresponding author: Chwen Jen Chen, cjchen@fcs.unimas.my Australasian Journal of Educational Technology © 2013. Please cite as: Chen, C. J., & Teh, C. S. (2013). Enhancing an instructional design model for virtual reality-based learning. Australasian Journal of Educational Technology, 29(5), 699-716. Australasian Journal of Educational Technology, 2013, 29(5). 715 Appendix: Instruments for the one-to-one evaluation Micro-strategy Interview guide • [Introduce yourself by name.] • Thank you for agreeing to help with this study. • Today you are going to evaluate a virtual reality (VR)-based learning system (NoviCAD) • NoviCAD is developed for helping novice car drivers to learn about various traffic rules and traffic signs. • In this session, I will give you some tasks to perform. I want you to help finding out what problems NoviCAD poses and how it could be improved in a way that you think can help you better learn the traffic rules • In the process of solving each task, we want you to voice your thoughts o How do you solve the task? o Why? o What you think? • [For each variable, the experimenter is to switch the learning system between the varied element and check whether the user notice the differences.] • It is the system that is being tested and definitely, not you. • [Start recording the session.] Debriefing a) Would you really use this system if you had a choice? b) What are the strengths of this system? c) What are the weaknesses of this system? d) If you would change only one thing in this system, what would it be? e) Do you have any suggestions to improve the system so that it can better help your learning? f) Are there any other comments of the system? Macro-strategy Interview Questions Problem Context Request and ensure the participant explores the learning system according to the following sequence. Guide him/her from item 1 to 4 and as for item 5, request him/her to explore carefully from 'Senario 1' [Scenario 1] until 'Senario 5' [Scenario 5]. 1. 'Pengenalan' [Introduction] 2. 'Kepentingan' [Importance] 3. 'Matlamat' [Aim] After reading these pages, are you motivated to know more or further explore the learning system? 4. 'Masalah Pembelajaran' [Learning Problem] After reading this 'Masalah Pembelajaran' [Learning Problem], are you excited to resolve it? 5. Explore 'Senario 1' [Scenario 1] until 'Senario 5' [Scenario 5] in the exact sequence. When the participant completes this task, ask the following questions. Australasian Journal of Educational Technology, 2013, 29(5). 716 Problem Representation Do you think such three-dimensional virtual road scenario (point at the virtual environment) can help you to understand traffic rules & traffic signs? If yes, why do you think it can help you to understand traffic rules & traffic signs? If no, why do you think it cannot help you to understand traffic rules & traffic signs? Do you think this virtual road scenario is sufficiently real to help your learning? If no, how do you think it should be improved? Problem Manipulation Space Do you think your interaction and navigation within the virtual road scenario help in your learning process? If yes, why do you think such interaction and navigation help? If no, why do you think such interaction and navigation do not help? Do you face any difficulty in navigating through the virtual environment? If yes, how do you think this navigation can be improved? Information Resources There are a few hyperlinks for each scenario (point at them). Do you take the effort to refer to them? If yes, is it obvious that you should refer to the links? If no, why you did not refer to them? How to make them more obvious? Coaching Are the pop-up messages helpful? If yes, why do you think these messages are helpful? If no, why do you think these messages are not helpful? Are these messages effective in capturing your attention? If yes, how do the messages capture your attention? If no, what changes should be made to the messages so that they can capture your attention? Did you seriously read or listen to these messages? Why? Scaffolding Comparing 'Senario 1' [Scenario 1] and other scenarios (e.g., 'Senario 5' [Scenario 5]), which one is the simplest or easiest to accomplish? Do you think sequencing the scenarios from the simple to complex helps in your learning? Why? Constructivist In this learning system, you have learned some traffic rules and traffic signs by solving a problem (going from one place to another). Do you like this way of learning or you prefer to learning system to inform you directly about these rules and signs? Why?