Microsoft Word - chen.doc


Australasian Journal of
Educational Technology

2006, 22(1), 39-63

The design, development and evaluation of a
virtual reality based learning environment

Chwen Jen Chen
Universiti Malaysia Sarawak

Many researchers and instructional designers increasingly recognise the
benefits of utilising three dimensional virtual reality (VR) technology in
instruction. In general, there are two types of VR system, the immersive
system and the non-immersive system. This article focuses on the latter
system that merely uses the conventional personal computer setting.
Although VR is recognised as an impressive learning tool, there are still
many issues that need further investigations. These include (i) identifying
the appropriate theories and/or models to guide its design and
development, (ii) investigating how its attributes are able to support
learning, finding out whether its use can improve the intended performance
and understanding, and investigating ways to reach more effective learning
when using this technology, and (iii) investigating its impact on learners
with different aptitudes. This project chose a learning problem that was
related to novice car driver instruction, to study some aspects of these
issues. Indeed, the study provided valuable insights to a feasible
instructional design theoretical framework, as well as an instructional
development framework for VR based learning environments. In addition, it
also developed understanding of the educational effectiveness of such a
learning environment and its effect on learners with different aptitude.

Introduction

Three dimensional virtual reality (VR) is one of the latest innovations in the
long line of technologies that can be used for teaching and learning.
Although VR seems to offer promising instructional benefits, there are still
many issues that need further investigation. Gustafson (2002) classifies low
cost three dimensional visualisation and "walk around" VR as a few of the
technologies that will revolutionise how people of all ages learn and work,
and Rieber (1996) assures that this technology has the potential to facilitate
the acquisition of higher order thinking and problem solving skills.
However, they also believe that much exciting research and development
work remains to be done.



40 Australasian Journal of Educational Technology, 2006, 22(1)

As with any other technological advancement, the introduction of this
technology brings about excitement and high expectation among educators
of its capabilities. However, it is important to note that this technology is
merely a tool, as is a chalkboard, television, overhead projector, or an
Internet connection. Tools by themselves do not teach. They have to be
carefully and effectively implemented to assist in the learning process. As
pointed out by Reigeluth and Frick (1999), more instructional design
theories are needed to provide guidance on the use of new information
technology tools. Hence, the pertinent questions would be on how to
design instruction to enable the effective utilisation of VR capabilities to
support the desired outcomes. What are the appropriate theories and/or
models to guide the design and development of such learning
environments?

Youngblut (1998) who has done a rather comprehensive review of
educational uses of VR technology reveals that most educational
applications are not thoroughly evaluated in terms of their educational
effectiveness. As pointed out by Jonassen, Hernandez-Serrano and Choi
(2000), technologies were previously employed for communicating
knowledge. However, in recent years, technologies are reconceived as
contexts, productivity tools, and thinking tools. In view of the fact that
effective learning is the ultimate aim of any teaching and learning efforts
and the roles of new technologies in teaching and learning are changing, it
is thus crucial to perform educational effectiveness evaluation. Such
evaluation, in the case of novice car driver instruction, intends to find out
whether the use of a three dimensional VR based learning environment can
improve the intended performance and understanding, and to investigate
how the attributes of this technology are able to support learning in a way
that conventional instructional methods are unable to provide (Kozma,
1994). This is succinctly articulated by Roblyer and Knezek (2003), who
suggest that technologies should not be viewed as delivery systems, rather
as components of solutions to educational problems.

Purpose

This project aimed to pursue development goals (Reeves, 2000) by focusing
on the dual objectives of developing a plausible solution to solve a learning
problem in a real context, while at the same time constructing a feasible
instructional design and development framework that could guide future
developmental efforts. It started by addressing a learning problem in a real
context and, as elaborated earlier, the identified learning problem was
related to the novice car driver instruction program in Malaysia. This
learning problem focused on the understanding of traffic rules for various
road scenarios including ordinary roads, different types of road junctions,
and related traffic signs. Then, a plausible solution to this learning problem



Chen 41

was designed and developed by integrating known and hypothetical
design principles with technological affordances. A recursive and reflective
approach was employed to evaluate and refine the solution. Finally, an
evaluation that employed a quasi-experimental design was conducted to
investigate the effects of the developed learning environment on learning,
and to investigate the effects of learners’ spatial visualisation abilities on
learning.

Spatial visualisation is a measure of the ability to mentally restructure or
manipulate the components of the visual stimulus and involves
recognising, retaining and recalling configurations when the figure or parts
of the figure are moved (McGee, 1979). In line with this, VR is known as a
promising medium for teaching people about the spatial characteristics of
places and situations due to its inherent spatial nature. To date, many
studies have been conducted to investigate the interaction of individuals’
spatial abilities with the characteristics of VR. Hence, a focus of the
evaluation study of this project is to further investigate this issue in an
effort to adapt the nature of VR based instruction to accommodate
individuals of different spatial abilities.

The following are the specific objectives for this project:

• to analyse a learning problem in a real context, in collaboration with
subject matter experts and by reviewing the related materials. The
identified learning problem concerned novice car driver instruction in
Malaysia, focusing solely on the cognitive aspect.

• to design a solution for the learning problem, which was a VR based
learning environment, by aligning the technological affordances with
the instructional design theoretical framework.

• to develop the VR based learning environment.

• to conduct a field test, which was a pre-test, post-test quasi-
experimental study comparing two learning modes (VR mode utilised
the VR based learning environment and Non VR mode utilised the
conventional learning methods) to measure the effects of each learning
mode on learning.

• to conduct an aptitude by treatment interaction (ATI) study to
investigate the effects of learners’ spatial visualisation abilities and the
possible interaction effects between the two learning modes and this
aptitude.



42 Australasian Journal of Educational Technology, 2006, 22(1)

Methodology

This section first provides a description of the learning problem. It then
describes in detail the methodology for the design, development and
evaluation of the VR based learning environment for this learning problem.

Analysis of the learning problem

As reported in a local newspaper item, Computer based traffic rules and
regulations test: More than 200,000 failures (in Chinese, 28 November 2003),
from May 2002 until August 2003, about 500,000 candidates undertook the
Road Transport Department (RTD) computer based theory test. However,
only 219,965 passed the test. Further investigation revealed that those who
did not pass the test had difficulty in studying the materials, which is in the
form of text and two dimensional static images.

Indeed, the current methods of instruction for the cognitive domain, which
rely on textbook and basic practical lessons, are observed to pose various
limitations in assisting learners in recalling or recognising knowledge, and
developing their understandings, intellectual abilities and skills. These
include supporting learners with different cognitive abilities, providing
authentic and meaningful tasks, providing concrete experience and active
experimentation to support this type of learning style, and providing
learner directed learning. On the other hand, the various capabilities of the
VR technology are foreseen to be able to overcome the observed problems,
such as its ability to provide three dimensional representation, authentic
representation, an excellent visualisation tool, multiple perspectives,
controlled level of complexity, active learning, learner centred, and strong
positive emotional reaction. Chen, Toh and Wan (2003) provide a thorough
description of these limitations and highlight the potential of VR to
overcome them.

Design, development and evaluation

The following describes the instructional design theoretical framework and
each activity of the instructional development model that was employed in
developing the VR based learning environment.

Instructional design theoretical framework
The instructional design theoretical underpinnings provided guidance
about the methods for facilitating learning. They provided guidelines on
determining the design of the VR based learning environment.

An eclectic approach uses a combination of principles from different
theories (Alessi & Trollip, 2001). This project took such an approach by



Chen 43

combining the concept of integrative goals, as proposed by Gagné and
Merrill (1990), with the model for designing constructivist learning
environments, as proposed by Jonassen (1999). They served as the macro-
strategy, which according to Reigeluth and Merrill (1978) is concerned with
the selection, sequence, and organisation of the subject matter topics that
are to be presented. A constructivist paradigm was chosen as it is in accord
with the new paradigm of instruction, and more important, VR capabilities
are found to be compatible with the constructivist learning principles
(Bricken, 1990; Chen & Teh, 2000; Winn, 1993). Additionally, the cognitive
theory of multimedia learning, from which a number of principles for the
design of multimedia messages are derived (Mayer, 2002), served as the
micro-strategy that basically, is concerned with presentation strategies.
Figure 1 depicts the instructional design theoretical framework, and Chen,
Toh and Wan (2004) provided the details of how this theoretical framework
guides the design of the learning environment.

Figure 1: Instructional design theoretical framework
of the VR based learning environment

Instructional development model
The instructional development model concerns what process an
instructional designer should use to plan and prepare for the instruction
(Reigeluth, 1999). The learning environment of this project (Figure 2 shows
a screenshot) adopted the Recursive, Reflective Design and Development
(R2D2) model. This constructivist instructional development model was
put forward by Willis (1995) and later revised in Willis and Wright (2000).
Three important but flexible guidelines for R2D2 model include (i)
recursive, non-linear design; (ii) reflective design; and (iii) participatory
design (Willis, 2000) This model was used to guide the design,
development and evaluation of the learning environment. It has three focal

Support tools

+

Micro-strategy

Enterprise scenario/
Problem

Objectives

Integrative goal
Principles

of multimedia
design

Macro-strategy



44 Australasian Journal of Educational Technology, 2006, 22(1)

points: Define, Design and Development, and Dissemination. The details of
the work done in the Define focal point and the Design and Development
focal point are reported in Chen and Toh (2005). This paper focuses on the
Dissemination focal point.

Figure 2: A screenshot of the VR based learning environment depicting the
incorporation of problem, cognitive tools and instructional activities

Dissemination focus

This focal point comprises four activities: summative evaluation,
packaging, diffusion, and adoption. However, this project looked into the
first activity, which was a summative evaluation to measure the outcomes
of each learning mode.

The study employed a pre-test, post-test quasi-experimental design. This
design involved two groups, VR and Non VR. Each group was given a pre-
test and a post-test. However, the groups did not have pre-experimental
sampling equivalence. The groups constituted intact classes, in which
equivalency could not be presumed or assured.



Chen 45

Population and sample

The current law in this country allows any person who is 17 years old and
above to undergo the novice car driver instruction program. Although the
ages of these candidates may vary greatly, according to Mohd Kifli, the
former Head of the Driving License Unit at Penang RTD, the majority are
from the younger group, those who are just over the eligible age (personal
communication, 25 February 2003). Hence, only individuals from this
group of population were chosen to evaluate the VR based learning
environment, which also implied that the usability of the learning
environment only took into account the cognitive abilities of this group of
the population. This majority formed the targeted population of this study.

Due to time and cost constraints, the accessible population for this study
encompassed Form Four students only (limited to those who had not
undergone the driver instruction program) of any Penang Island secondary
schools that were well-equipped with multimedia computer laboratories.
Form Four students were chosen because they were a non-examination
class and more important, they were within the targeted population as
their age was approximately the minimum eligible age to undergo the
novice car driver instruction program. School students were chosen instead
of the general public, in order to obtain better controlled samples. The
sample size was 126 and the average age of the participants was 16.55
years.

Three secondary schools were randomly selected (based on the simple
random sampling technique) from the list of daily secondary schools on
Penang Island. For each school, three intact classes were randomly chosen.
All eligible students (those who had not undergone the driver instruction
program) in the selected classes were included in the study. These selected
classes were randomly assigned to either VR group or Non VR group.

Material

The VR based learning environment served as the treatment for the VR
group while the Non VR group utilised the conventional learning method
that relied on lectures and reading materials.

Instruments

The instruments that were used include the VR based test (pre-test and
post-test) and the Bennett, Seashore and Wesman Space Relations Test.

VR based test (pre-test and post-test)
The VR based pre-test and VR based post-test that were employed in this
study were computer based. Each test consisted of 15 questions and aimed



46 Australasian Journal of Educational Technology, 2006, 22(1)

to assess the learners’ understanding of traffic rules and traffic signs.
Unlike the conventional theory test, set by RTD, that showed two
dimensional images, each of the questions in the VR based test showed a
three dimensional simulation of a virtual road scenario and the learners
were instructed to identify an observable error within the simulation, if
any.

Nyatakan kesalahan lalulintas yang anda dapat kenalpasti (jika ada) dalam
simulasi yang ditunjukkan. Tekan ’F5’ untuk ulangi demonstrasi, jika perlu.

A. Kenderaan biru telah menggunakan lorong yang tidak betul setelah
membelok keluar dari persimpangan.

B. Kenderaan biru yang menggunakan lorong kiri jalan sebelum tiba di
persimpangan harus juga membelok keluar ke kiri di persimpangan.

C. Kenderaan putih harus memberi laluan kepada kenderaan biru.
D. Tiada kesalahan.

Figure 3: A sample question of the VR based
test that is related to traffic rules

Both pre-test and post-test were similar in content but the order of the
questions was different to avoid the set response effect. A subject matter
expert from the RTD was requested to review the process used to develop
the test, as well as the test itself, and then made a judgment about how well
these items represent the intended content area. The Cronbach’s alpha
reliability coefficient was 0.83, which depicted the test items were
satisfactorily reliable. Figure 3 depicts a screenshot of a test item.

Road scenario simulation



Chen 47

The maximum score for each test was 15. For each question, participants
received a score of either 1 (correct answer) or 0 (incorrect answer), and a
total score ranging from 0 to 15. This total score was multiplied by 100 to
convert it to a percentage. There were six traffic signs related questions and
nine traffic rules related questions.

Bennett, Seashore and Wesman Space Relations Test
This instrument was chosen to test the spatial visualisation ability of the
participants. It consists of 60 patterns, which can be folded into figures. A
feature inherent in these items is that they required mental manipulation of
objects in three dimensional space. It tests the ability to visualise a
constructed object from a picture of pattern, which is illustrated in two
dimensional. This is consistent with the spatial skill needed to reconstruct
the three dimensional view of the road scenario from the two dimensional
plan view. This test has a reliability of 0.91 (Bennett, Seashore & Wesman,
1972). Participants who score 50% or above are classified as having high
spatial visualisation ability and participants who score 50% or below are
classified as having low spatial visualisation ability.

Procedure

Two weeks before the treatment, the learners were given the Bennett,
Seashore and Wesman Space Relations Test and the VR based pre-test.
Then, immediately before the treatment, the experimental group was given
training on the navigation of the virtual environment. Immediately after
the treatment, the learners were given the VR based post-test.

Results

Distribution of learners

The 126 learners were divided into two groups. Each group was assigned
to one of the two learning modes. A total of 62 learners were assigned to
the VR mode while 64 learners were assigned to the Non VR mode.

Testing of Hypothesis 1

H1: Learners exposed to the VR mode will have evidence of better
learning than those exposed to the Non VR mode.

A one way analysis of covariance (ANCOVA) was conducted to compare
the effects of the two modes on learning. This analysis was meant to
determine if there was statistically significant difference in the adjusted
mean of the dependent variable (gain score) between the two learning
modes, while controlling the pre-test. The independent variable was the



48 Australasian Journal of Educational Technology, 2006, 22(1)

learning mode (VR and Non VR) and the dependent variable was the gain
score, which was obtained by subtracting each post-test score from the
respective pre-test score. Learners’ scores on the pre-test were used as the
covariate in the analysis. Preliminary checks were conducted to ensure that
there was no violation of the assumptions of normality, linearity,
homogeneity of variances, homogeneity of regression slopes, and reliable
measurement of the covariate.

Table 1: One way ANCOVA of gain score by learning mode with
pre-test score as covariate. Dependent variable: Gain score

Source Type III SS df MS F Sig. eta2

Covariate
Pre-test score 12919.724 1 12919.724 67.182 0.000 0.353
Main effect
Learning mode 6639.222 1 6639.222 34.524 0.000 0.219
Error 23654.133 123 192.310
Total 100711.111 126
p < 0.05

After adjusting for the pre-test scores, there was a significant difference
between the two learning modes on the gain scores, F(1, 123) = 34.524, p =
0.000 (see Table 1). The effect size, calculated using eta2, was 0.219, which in
Cohen’s (1988) terms would be considered a large effect size. Cohen
classified 0.01 as a small effect, 0.06 as a medium effect and 0.14 as a large
effect. There was also a strong relationship between the pre-test scores and
the gain scores, as indicated by the eta2 of 0.353. The VR mode had an
adjusted mean, M, of 28.751 and the Non VR mode had an adjusted mean,
M, of 14.231. Thus, it could be concluded that learners exposed to the VR
mode obtained significantly higher gain score for the VR based test than
learners exposed to the Non VR mode.

Testing of Hypothesis 2

H2: Low spatial visualisation ability learners exposed to the VR
mode will have evidence of better learning than low spatial
visualisation ability learners exposed to the Non VR mode.

A one way ANCOVA was conducted to examine if there was significant
difference in adjusted means of the dependent variable (gain score)
between the low spatial visualisation ability learners of both learning
modes, while controlling the pre-test. After adjusting for the pre-test
scores, there was a significant difference between the low spatial
visualisation ability learners of the two learning modes on the gain scores,
F(1, 57) = 13.129, p = 0.001 (see Table 2). This means that the learning mode
had a main effect on the low spatial visualisation ability learners’ gain
scores. The effect size, calculated using eta2, was 0.187, which in Cohen’s



Chen 49

(1988) terms would be considered a large effect size. This effect size
indicated that the learning mode effect accounted for 18.7% of the variance
of the low spatial visualisation ability learners’ gain scores.

Table 2: One way ANCOVA of gain score by learning mode
with pre-test score as covariate for low spatial visualisation

ability learners. Dependent variable: Gain score.

Source Type III SS df MS F Sig. eta2

Covariate
Pre-test score 7718.155 1 7718.155 32.489 0.000 0.363
Main effect
Learning mode 3118.906 1 3118.906 13.129 0.001 0.187
Error 13541.105 57 237.563
Total 54577.778 60
p < 0.05

The low spatial visualisation ability learners of the VR mode had higher
adjusted mean (adjusted M = 29.894) than the low spatial visualisation
ability learners of the Non VR mode (adjusted M = 15.440). Hence, it could
be concluded that the low spatial visualisation ability learners exposed to
the VR mode obtained significantly higher gain score for the VR based test
than the low spatial visualisation ability learners exposed to the Non VR
mode.

Testing of Hypothesis 3

H3: High spatial visualisation ability learners exposed to the VR
mode will have evidence of better learning than high spatial
visualisation ability learners exposed to the Non VR mode.

Table 3: One way ANCOVA of gain score by learning mode
with pre-test score as covariate for high spatial visualisation

ability learners. Dependent variable: Gain score.

Source Type III SS df MS F Sig. eta2

Covariate
Pre-test score 5129.109 1 5129.109 32.440 0.000 0.340
Main effect
Learning mode 3605.483 2 3605.483 22.803 0.000 0.266
Error 9961.087 63 158.112
Total 46133.333 66
p < 0.05

A one way ANCOVA was conducted to examine if there was a significant
difference in adjusted mean of the dependent variable (gain score) between
the high spatial visualisation ability learners of the two learning modes,
while controlling the pre-test. After adjusting for the pre-test scores, there



50 Australasian Journal of Educational Technology, 2006, 22(1)

was a significant difference between the high spatial visualisation ability
learners of the two learning modes on the gain scores, F(1, 63) = 22.803, p =
0.000 (see Table 3). This means that the learning mode had a main effect on
the high spatial visualisation ability learners’ gain scores. The effect size,
calculated using eta2,  was 0.266, which in Cohen’s (1988) terms would be
considered a large effect size. This effect size indicated that the learning
mode effect accounted for 26.6% of the variance of the high spatial
visualisation ability learners’ gain scores.

The high spatial visualisation ability learners of the VR mode had a higher
adjusted mean (adjusted M = 27.832) than the high spatial visualisation
ability learners of the Non VR mode (adjusted M = 13.021). Hence, it could
be concluded that the high spatial visualisation ability learners exposed to
the VR mode obtained significantly higher gain scores for the VR based test
than the high spatial visualisation ability learners exposed to the Non VR
mode.

Testing of Hypothesis 4

H4: In the VR mode, high spatial visualisation ability learners will
have evidence of better learning than low spatial visualisation ability
learners.

Table 4: ANCOVA of gain score by spatial visualisation ability with pre-
test score as covariate for VR mode. Dependent variable: Gain score

Source Type III SS df MS F Sig. eta2

Covariate
Pre-test score 6054.738 1 6054.738 23.589 0.000 0.286
Main effect
Spatial visualisation ability 47.176 1 47.176 0.184 0.670 0.003
Error 15143.781 59 256.674
Total 72311.111 62
p < 0.05

A one way ANCOVA was conducted to examine if there was significant
difference in adjusted mean of the dependent variable (gain score) between
low spatial visualisation ability learners and high spatial visualisation
ability learners, while controlling the pre-test. Learners’ scores on the pre-
test were used as the covariate in the analysis. After adjusting for the pre-
test scores, it was found that there was no significant difference in the
adjusted mean gain scores for the high and low spatial visualisation ability
learners of the VR mode, F(1, 59) = 0.184, p = 0.670 (see Table 4).

The adjusted mean for the high spatial visualisation ability learners was
29.561 and the adjusted mean for the low spatial visualisation ability



Chen 51

learners was 27.801. The adjusted mean difference of 1.76 was not
significant. Thus, it could be concluded that the adjusted mean for the gain
score between the high spatial visualisation ability VR mode learners and
the low spatial visualisation ability learners of the same mode did not
differ significantly.

Testing of Hypothesis 5

H5: There is an interaction effect between the learners’ spatial
visualisation ability and the learning mode, related to gain score of
the VR based test.

A 2 by 2 two way ANCOVA was conducted to examine the effects of the
learning modes on the performance in the VR based test for low spatial
visualisation ability learners and high spatial visualisation ability learners.
The independent variables were the learning mode (VR and Non VR) and
spatial visualisation ability (low and high). The dependent variable was the
gain score of the VR based test and learners’ scores on the VR based pre-
test were used as the covariate in the analysis.

Table 5: Two way ANCOVA of gain score by learning mode
and spatial visualisation ability with pre-test score as

covariate. Dependent variable: Gain score

Source Type III SS df MS F Sig. eta2

Covariate
Pre-test score 12770.023 1 12770.023 65.531 0.000 0.351
Main effects
Learning mode (LM)
Spatial visualisation ability
(SVA)

6636.703
73.624

1
1

6636.703
73.624

34.057
0.378

0.000
0.540

0.220
0.003

2 way interaction
LM SVA 1.125 1 1.125 0.006 0.940 0.000
Error 23579.433 121 194.871
Total 100711.111 126
p < 0.05

The two way ANCOVA results, as shown in Table 5, indicated that the
interaction between learning mode and spatial visualisation ability was not
significant, F(1, 121)=0.006, p < 0.940. This means the differences in the
adjusted means of the gain score between the two learning modes did not
vary as a function of learners’ spatial visualisation abilities. Although the
effect of the learning modes on the gain scores of the VR based test did not
depend on the spatial visualisation ability level, there were differences in
gain scores among the learning modes for learners of different spatial
visualisation abilities. The VR mode had higher gain score than the Non
VR mode for both the low and high spatial visualisation ability learners.



52 Australasian Journal of Educational Technology, 2006, 22(1)

The results in Table 5 also show that the main effect due to spatial
visualisation ability was not statistically significant, F(1, 121)=0.378, p <
0.540. The adjusted mean of the gain score for the high spatial visualisation
ability learners, averaged across the two learning modes, 22.234, was not
significantly higher than the adjusted mean of the gain score for the low
spatial visualisation ability learners, 20.677. Table 6 presents the means,
standard deviations, adjusted means, and standard error of the gain score
by learning mode and spatial visualisation ability.

Table 6: Means, standard deviations, adjusted means, and standard errors
of gain score by learning mode and spatial visualisation ability

Gain scoreLearning
mode

Spatial visualisation
ability M SD Adjusted M SE

Low (N=30) 29.1111 21.6898 27.850a 2.553VR
High (N=32) 28.3333 15.6118 29.597a 2.473
Low (N=30) 16.2222 16.2059 13.504a 2.571Non VR
High (N=34) 12.5490 15.1103 14.872a 2.411

Note: a Evaluated at covariate appeared in the model: pre-test = 57.6720

Thus, the statistical results depicted that there was no significant
interaction between the learning modes and the learners’ spatial
visualisation abilities. This means the effect of the learning modes on the
gain scores of the VR based test did not depend on the spatial visualisation
ability levels. The main effect due to spatial visualisation ability was also
not statistically significant.

Discussion

VR mode versus Non VR mode

Learners exposed to the VR mode significantly outperformed the learners
exposed to the Non VR mode. The cognitive theory of multimedia learning
(Mayer, 2002), with its underlying cognitive load theory (Sweller, 1999) and
dual coding theory (Paivio, 1986), help to explain this finding. This is
elaborated below.

A possible explanation for this finding is related to the prediction that
overloading the working memory results in less effective learning. Most
spatial problems require learners to deal with the construction of three
dimensional mental representations (Carpenter & Just, 1986; Cooper, 1988;
Keenan & Moore, 1979) in order to understand and manipulate given
spatial information. The content and structure of mental representation of
spatial information, and how easily and accurately it is constructed and
maintained during processing, is central to dealing successfully with
spatial problems (Carpenter & Just, 1986). According to Pillay (1997; 1998),



Chen 53

the construction and understanding of three dimensional representations
involves complex cognitive activities, such as integrating information from
various sources, constructing meaning from abstruse elements in the
graphical information, constructing hidden or obscured information,
constructing and maintaining three dimensional representations of the
components and of the complete object, retrieving relevant schemas,
encoding or constructing information from given spatial representations,
and finally, performing transformations on these representations.

The learning problem that was chosen in this project is an example of a
spatial problem. Learners of the VR mode outperformed the learners of the
Non VR mode because in addition to text and image materials, learners of
the VR mode were also provided with virtual environments. The use of
virtual environments providing three dimensional representations that
mimic various real world road scenarios, offer virtual experience of
navigating through them, and show the movement of vehicles in the VR
mode, primarily eliminate the learners’ need to mentally construct and
maintain the three dimensional representation and dynamics of the road
scenario. On the contrary, learners in the Non VR mode need to involve
more complex cognitive activities. These include integrating information
from text and two dimensional plan views regarding the road scenarios,
constructing meaning of abstruse elements in the graphical and text
information, particularly on the dynamics of the virtual vehicles,
constructing the depth information of the road scenario, which is
unavailable when presented using two dimensional plan views,
constructing and maintaining the three dimensional representation and
dynamics of the road scenarios, retrieving relevant schema to support these
cognitive activities, and encoding information from the formed spatial
representations.

Sweller (1999) argues that the cognitive load imposed by task information
may exhaust the cognitive resources of individuals and consequently affect
learning. In the Non VR mode, learners are required to involve complex
cognitive activities by directing attention to multiple sources of
information, and to synthesise these before proceeding any further. The
need to integrate information from multiple sources imposes extraneous
cognitive load that may inhibit learning (Chandler & Sweller, 1992).
Sweller (1999) also argues that for learning to occur, learners must attend to
essential concepts associated with the task and must develop schemas.
Attention must be directed to learning the essential concepts and not to
extraneous processes. Many of the cognitive activities that occur in the Non
VR mode as mentioned earlier are presumably imposing extraneous
demands on limited cognitive resources, Indeed, many of these activities
are not essential for learning the content, but become necessary due to the
instructional format. As the cognitive capacity is limited, the use of



54 Australasian Journal of Educational Technology, 2006, 22(1)

cognitive resources for extraneous cognitive demands may cause
insufficient resources available for learning (Sweller, 1999). On the other
hand, the incorporation of virtual environments into the VR mode provides
a more automated spatial encoding and therefore does not require specific
spatial processing schema to mentally construct a three dimensional view
from the respective two dimensional plan view, which occurs in the Non
VR mode. Based on the cognitive load theory (Sweller, 1999), the amount of
mental resources that deal with extraneous cognitive load can be reduced
with the availability of such support, thus leaving more resources for
intrinsic cognitive load. This most probably explains the significant
positive effects of the VR mode when compared with the Non VR mode.

Learners of the Non VR mode were exposed to materials in the form of text
and two dimensional image, but were tested on the VR based test, which
posed questions that showed three dimensional simulation of virtual road
scenarios. The dissimilarity in the representations between the learning
environment and the test poses the need for the Non VR learners to use
another level of abstraction when answering the VR based test. Based on
the cognitive load theory (Sweller, 1999), this once again imposes a higher
level of extraneous cognitive load on the Non VR learners, particularly for
those who do not have the necessary schema for spatial processing in the
long term memory. As a result, learners in this mode performed less well
than their VR counterparts, in the VR based test. The following provides
several possible explanations based on dual processing.

Paivio (1971, 1986) was among the first to demonstrate experimentally that
spatial processing (in the form of mental imagery) can enhance the recall of
verbal information. Numerous studies show that the processing of adjunct
spatial information can enhance the comprehension and recall of the text
material (e.g. Alesandrini, 1984; Dean & Kulhavy, 1981; Rieber, 1994;
Schwartz & Kulhavy, 1981). Adjunct verbal processing is also shown to
enhance spatial learning (Kulhavy, Lee, & Caterino, 1985; Schofield &
Kirby, 1994). These enhancement or collaborative effects are interpreted in
terms of dual coding (Paivio, 1971, 1986).

The VR mode utilises mostly text, images, and virtual environments, which
also implies that this learning mode involves both verbal (text) and non-
verbal (images and virtual environments) processing. Based on the dual
coding theory, both verbal and non-verbal processing have equal weight
and as stated in Kirby (1993), the activation of this dual processing has
three advantages: (a) it results in a more elaborate encoding of the material
to be learned, providing increased retrieval cues; (b) more information can
be stored by dual coding, because verbal and spatial information are stored
separately, and (c) the alternative store may be optimal for retaining certain
types of information. These advantages are termed as enhancement or



Chen 55

collaborative effects and probably explain the significant positive effects of
the VR mode.

Despite the numerous studies that demonstrate the enhancement effects,
examples are found of tasks in which verbal and spatial processing
compete rather than collaborate (e.g., Kirby, Jurisich & Moore, 1984; Kirby,
1990). In such cases, dual processing does not enhance learning and is
known to produce competitive effects. The Non VR mode primarily
comprises text and images (2D plan views). As with the VR mode, Non VR
mode also involves both verbal (text) and non-verbal (images) processing
or both auditory and visual processing as defined in the cognitive theory of
multimedia learning. Nevertheless, this study found that the dual coding
or processing that occurred in the Non VR mode did not assist learning as
much as the dual processing that occurred in the VR mode.

Kirby (1993) identifies four boundary conditions when dual coding is likely
to result in collaborative or competitive effects: task interference, focus of
attention, nature of the information, and individual differences. In the case
of VR mode versus Non VR mode, the conditions on task interference and
nature of the information are found particularly relevant to explain the
finding reported earlier.

Dual processing requires two tasks to be performed simultaneously.
However, according to Kirby (1993), if the required verbal encoding and
non-verbal encoding are not automated, they will compete for the limited
cognitive resources. The greater the difficulty in performing each of the
tasks, the greater the interference will be, which will then lead to greater
competitive effects. The absence of virtual environments in the Non VR
mode that provide three dimensional representation, offer virtual
navigation experience and show the dynamics of the road scenarios, or the
reliance on two dimensional plan views that require learners to use
complex cognitive activities to construct and comprehend the three
dimensional representations produces a less automated spatial encoding.
As described in Kirby (1993), this potentially increases the cognitive load,
causes more difficulty in performing the spatial task and leads to greater
interference with the verbal task. On the hand, the VR mode produces less
cognitive load. This causes less interference between the verbal task and
the non-verbal task, which also infers collaboration instead of competition
between the two tasks.

Another boundary condition of dual coding that can explain this finding is
the ‘nature of the information’. Kirby (1993) points out that not all
information is equally amenable to both non-verbal and verbal coding.
Abstract verbal propositions are not easily coded non-verbally (e.g.
spatially), at least without loss of crucial information. Conversely, much



56 Australasian Journal of Educational Technology, 2006, 22(1)

non-verbal (e.g. spatial) information is difficult to code verbally. Too much
effort in encoding information will distract learners from other content
and/or cause interference. This learning problem, which is meant to help
learners to interpret various traffic rules, is spatial in nature. Thus, the use
of virtual environments to present spatial information in the VR mode,
which mimics the real world representation requires less encoding effort
and therefore assists more in the learning than the Non VR mode. On the
other hand, in the Non VR mode, much effort in encoding verbal (text) and
non-verbal (image instead of spatial) for information that is spatial in
nature, results in competitive effects.

Nevertheless, the significant positive effects of the VR mode may also be
due to the fact that learners in the VR mode learned traffic rules using the
VR based learning environment, and then were tested with items that
involved the same modality. As the VR simulations of various road
incidents in the test items portrayed much the same scenes as those used in
the learning session, the significant positive results of those in the VR mode
may thus reflect the benefits of this repetition rather than the learning of
traffic rules per se.

Effects of the learning mode on learning based on spatial
visualisation ability levels

The findings that both high and low spatial visualisation ability learners
benefit from the VR mode, and that the difference between the
performances of these two groups of learners is not significant, is
inconsistent with the studies by Mayer and Sims (1994) and Toh (1998).
These studies provide preliminary evidence that high spatial ability
learners benefit more from improved instructional design than do low
spatial ability learners. There are several possible explanations for this
finding.

First, the use of virtual environments that explicitly present the three
dimensional representations and dynamics of the road scenarios, and the
use of additional navigational aids to help learners to stay oriented in the
VR mode, have greatly reduced the learners’ need to use their existing
spatial processing schema. This reduces the extraneous cognitive load,
which subsequently enables the high spatial visualisation ability learners to
spare most of their cognitive resources for comprehending the content of
the learning environment. This possibly has explained the significant
positive effect of this learning mode on the high spatial visualisation ability
learners.

Second, in this learning mode, the lack of spatial processing schema among
the low spatial visualisation ability learners also has not negatively affected



Chen 57

the learning by this group. The explicit presentation of three dimensional
representation and dynamics of the road scenarios and the use of
additional navigational aids have kept the need for using this schema to a
minimum. Thus, the low spatial visualisation ability learners of the VR
mode significantly outperform their Non VR counterparts, as they are able
to use their cognitive resources for comprehending the content, rather than
using these resources to support the additional cognitive activities that are
necessary in the effort to comprehend the content, such as those occurring
in the Non VR mode.

The finding that the low spatial visualisation ability learners of the VR
mode significantly outperform their Non VR counterparts may also be
explained by Messick’s strategies for matching individual differences and
learning task. Messick (1976) proposed three strategies: the challenge
match, the capitalisation match, and the compensatory match. The
challenge match uses a deliberate mismatch between task demands and
learner capabilities, to force a learner to change and become more flexible.
The capitalisation match aims to tailor task demands to match the learners’
strengths, and the compensatory match aims to offset learners’ deficiencies
by providing cognitive tools that learners cannot readily provide for
themselves.

The VR mode that produces significant positive effects for both high and
low spatial visualisation ability learners employs the compensatory
matching technique. The use of virtual environments, which are
unavailable in the Non VR mode, compensates for the need to mentally
construct and maintain the three dimensional representation and dynamics
of the road scenarios. The non-significant difference between high spatial
visualisation ability learners and low spatial visualisation ability learners of
the VR mode proves the success of the compensatory match. Stanney and
Salvendy (1995), in their study, also reveal that the performance gap
between high spatial individuals and low spatial individuals disappears
with the use of interface design that can avoid the need for mentally
structuring embedded information. This interface improves the
performance of low spatial ability learners due to the compensatory match
of their abilities.

Interaction effects
Another finding from this study is that the interaction effect between the
learners’ spatial visualisation abilities and the two learning modes is not
significant. This implies that the effects of the learning modes on learning
do not depend on the learners’ differences in terms of spatial visualisation
ability. However, between the two learning modes, VR mode provides
more positive effect to both low spatial visualisation ability learners and
high spatial visualisation ability learners. Indeed, both low spatial



58 Australasian Journal of Educational Technology, 2006, 22(1)

visualisation ability learners and high spatial visualisation ability learners
benefit equally from this learning mode. This implies that the learners
benefit from the VR mode, irrespective of their spatial visualisation
abilities.

Implications of the project

There are four major implications of this project as elaborated below.

Feasible instructional design theoretical framework

Figure 1 illustrates the overall instructional design theoretical framework
that guided the design of the VR based learning environment. This
framework could serve as a feasible and useful template to guide the
design of other VR based learning environments, particularly learning
environments that adopt the new paradigm of instruction (constructivist
based). The fact that a robust model for guiding the design of learning
environment using this technology is still unavailable at present, this
framework functions as an initial structure which can be further refined
and/or revised in the effort to generate a robust design model for such
learning environment.

Feasible constructivist instructional development model

Virtual environment technology is noted as capable of affording
constructive learning (Bricken, 1990; Chen & Teh, 2000; Greening, 1998;
Jonassen et al., 2000; Winn, 1993), and a comprehensive review of
educational uses of virtual environment technology by Youngblut (1998)
reveals that applications that explicitly embody some form of pedagogy all
support constructivist learning. Although many of these applications make
known the employed philosophy of pedagogy, very few of them report on
the adopted instructional development models. In this regard, the
successful development of the learning environment in this project
provides evidence on the feasibility to employ a constructivist instructional
model (R2D2 model) to guide the design and development process of VR
based learning environments. Such model can also be used to guide the
design and development process of other VR based learning environments
and will be particularly appropriate for learning environments that employ
the constructivist paradigm.

VR based learning environment – an effective alternative

The significant positive effects of the VR based learning environment
(referring to the VR mode) on learning when compared with the Non VR



Chen 59

mode provides another evidence of the potentials of VR technology for
instructional use. This VR based learning environment provides new
learning opportunities by introducing learning activities that make visible
concepts and relationships that are not easily grasped or visualized by
learners when relying only on the conventional method. The feasibility of
implementing this VR based learning environment in terms of technical,
economic, and social, and the proven effectiveness of this method over the
conventional method provide strong justifications to suggest the RTD of
Malaysia to consider incorporating this novel learning method into the
current novice car driver instruction programme.

ATI study

The vast majority of the research into virtual environments for instructional
use is technology driven, rather than taking into account the human factor.
There has been little study on how learner characteristics interact with the
features of virtual environments either to aid or inhibit learning. The ATI
study that was conducted provides more understanding of this aspect. The
findings of this study show that learners benefit most from the VR mode,
irrespective of their spatial visualisation abilities. This proves the VR based
learning environment offers a promising medium to accommodate
individual differences in terms of this aptitude.

Recommendations and conclusion

This project raises a couple of other interesting issues that worth further
investigations. First, in the effort to further refine and/or revise the design
and development framework (both instructional design theoretical
framework and instructional development model) more design and
development of various types of VR applications based on the proposed
framework are needed. This will lead to the generation of a more robust
and comprehensive design and development framework for VR based
learning environments. Second, the interaction effects of VR based learning
environment can be extended to other differences among the learners, such
as locus of control, types of personality, field dependency, gender, motion
sickness history, and computer literacy. This will further enlighten us on
how learner characteristics interact with the features of virtual
environments to either aid or inhibit learning.

As a conclusion, this project proposes a feasible instructional design
theoretical framework that can be used to guide the design of other VR
based learning environments. This framework provides an initial structure
that can be further refined and/or revised in the effort to generate a robust
design model for such learning environment. The successful development



60 Australasian Journal of Educational Technology, 2006, 22(1)

of the VR based learning environment also proves the feasibility and
appropriateness to employ a constructivist instructional development
model (R2D2 model) to guide the design and development process of VR
based learning environments. The fact that real learning problem related to
the novice car driver instruction programme has been chosen in this project
and the significant positive effects of the VR mode over the Non VR mode,
which is employed in the current instruction programme, provides a
strong justification to the RTD that VR based learning environment makes
an effective alternative to the current method. The ATI study also shows
the benefit of the VR based learning environment in accommodating
individual differences in terms of spatial visualisation ability. In short, the
outcomes of this work provide another confirmation on the high potential
of VR technology for instructional use. It also points to the worthiness to
further investigate the methods that can improve the effectiveness of VR
based learning environments.

References
 
 Alesandrini, K.L. (1984). Pictures and adult learning. Instructional Science, 13, 63-77.
 

 Alessi, S.M. & Trollip, S.R. (2001). Multimedia for learning: Methods and development.
Massachusetts: Allyn & Bacon.

 

 Bennett, G.K., Seashore, H.G. & Wesman, A.G. (1972). Manual of differential aptitude
test. New York: The Psychological Corporation.

 

 Bricken, W. (1990). Learning in virtual reality. (Tech. Memo. M-90-5). Seattle, WA:
Human Interface Technology Laboratory, University of Washington.

 

 Carpenter, P.A. & Just, M.A. (1986). Spatial ability: An information processing
approach to psychometric. In R.J. Sternberg (Ed), Advances in the Psychology of
Human Intelligence, 3 (pp. 221-253). Hillsdale, NJ: LEA.

 

 Chandler, P. & Sweller, J. (1992). Split attention effects as a factor in the design of
instruction. British Journal of Educational Psychology, 62, 233-246.

 

 Chen, C.J. & Teh, C.S. (2000). An affordable virtual reality technology for
constructivist learning environments. The 4th Global Chinese Conference on
Computers in Education (pp. 414-421). Singapore.

 

 Chen, C.J., Toh, S.C. & Wan, M.F. (2003). Virtual reality: A potential technology for
providing novel perspective to novice driver education in Malaysia. In N.
Ansari et al. (Eds), Proceedings of the International Conference on Information
Technology: Research and Education (pp. 184-188). New Jersey: IEEE.

 

Chen, C.J., Toh, S.C. & Wan, M.F. (2004). The theoretical framework for designing
desktop virtual reality-based learning environments. Journal of Interactive
Learning Research., 15(2), 147-167.

 



Chen 61

 Chen, C.J. & Toh, S.C. (2005). A feasible instructional development model for
virtual reality (VR)-based learning environments: Its efficacy in the novice car
driver instruction of Malaysia. Educational Technology Research & Development,
53(1), 111-123.

 

 Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ:
Erlbaum.

 

 Computer-based traffic rules and regulations test: More than 200,000 failures (in
Chinese). (28 November 2003). Sin Chew Jit Poh. [viewed 28 Nov 2003].
http://www.sinchewi.com/print.phtml?sec=1 &artid=200311280278

 

 Cooper, L.A. (1988). The role of spatial representations in complex problem solving.
In S. Schiffer & S. Steele (Eds), Cognition and Representation (pp. 53-86). Boulder,
CO: Westview.

 

 Dean, R.S. & Kulhavy, R.W. (1981). Influence of spatial organization in prose
learning. Journal of Educational Psychology, 73, 57-64.

 

 Gagné, R.M. & Merrill, M.D. (1990). Integrative goals for instructional design.
Educational Technology Research & Development, 38(1), 23-30.

 
Greening, T. (1998). Building the constructivist toolbox: An exploration of cognitive

technologies. Educational Technology, 38(2), 28-35.
 
Gustafson, K.L. (2002). The future of instructional design. In R.A. Rieser & J.V.

Dempsey (Eds), Trends and issues in instructional design and technology. New
Jersey: Merrill Prentice Hall.

 
Jonassen, D.H. (1999). Designing constructivist learning environments. In C.M.

Reigeluth (Ed), Instructional-design theories and models: A new paradigm of
instructional theory (Vol. 2, pp. 215-239). New Jersey: Lawrence Erlbaum.

 
Jonassen, D.H., Hernandez-Serrano, J. & Choi, I. (2000). Integrating constructivism

and learning technologies. In M. Spector & T.M. Anderson (Eds), Integrated and
holistic perspectives on learning, instruction and technology: Understanding
complexity (pp.103-127). Netherlands: Kluwer Academic.

 
Keenan, J.M. & Moore, R.E. (1979). Memory for images of concealed objects: A

reexamination of Neisser and Kerr. Journal of Experimental Psychology: Human
Memory and Learning, 5, 374-385.

 
Kirby, J.R. (1990). Spatial cognition in education. International Congress of Applied

Psychology, Kyoto, Japan.
 

 Kirby, J.R. (1993). Collaborative and competitive effects of verbal and spatial
processes. Learning and Instruction, 3, 201-214.

 

 Kirby, J.R., Jurisich, R. & Moore, P.J. (1984). Effects of map processing upon text
comprehension. Annual meeting of the American Psychological Association, Toronto,
Canada.

 



62 Australasian Journal of Educational Technology, 2006, 22(1)

 Kozma, R.B. (1994). Will media influence learning? Reframing the debate.
Educational Technology Research & Development, 42(2), 7-19.

 

 Kulhavy, R.W., Lee, J.B., & Caterino, L.C. (1985). Conjoint retention of maps and
related discourse. Contemporary Educational Psychology, 13, 169-184.

 
Mayer, R.E. (2002). Multimedia learning. Cambridge: Cambridge University Press.
 
Mayer, R.E. & Sims, V.K. (1994). For whom is a picture worth a thousand words?

Extensions of a dual-coding theory of multimedia learning. Journal of Educational
Psychology, 86, 389-401.

 

 McGee, M.G. (1979). Human spatial abilities, sources of sex differences. New York:
Praeger.

 
Messick, S. (1976). Individuality in learning. San Francisco: Jossey-Bass.
 

 Paivio, A. (1971). Imagery and verbal processes. New York: Holt, Rinehart & Winston.
 
Paivio, A. (1986). Mental representations. New York: Oxford University Press.
 

 Pillay, H.K. (1997). Cognitive load and assembly tasks: Effects of instructional
formats on learning assembly procedures. Educational Psychology, 17(3), 285-299.

 

 Pillay, H.K. (1998). Cognitive process and strategies employed by children to learn
spatial representations. Learning and Instruction, 8(1), 1-18.

 
Reeves, T.C. (2000). Enhancing the worth of instructional technology research

through “design experiments” and other development research strategies.
Paper presented at International Perspectives on Instructional Technology Research
for the 21st Century. New Orleans, LA, USA. [verified 1 Mar 2006]
http://it.coe.uga.edu/~treeves/AERA2000Reeves.pdf

 
Reigeluth, C.M. & Frick, T.W. (1999). Formative research: A methodology for

creating and improving design theories. In C.M. Reigeluth (Ed), Instructional-
design theories and models: A new paradigm of instructional theory (pp. 633-652).
New Jersey: Lawrence Erlbaum.

 
Reigeluth, C.M. (1999). What is instructional-design theory and how is it changing?.

In C.M. Reigeluth (Ed), Instructional-design theories and models: A new paradigm of
instructional theory (pp. 5-29). New Jersey: Lawrence Erlbaum.

 
Reigeluth, C.M. & Merrill, M.D. (1978). A knowledge base for improving our

methods of instruction. Educational Psychologist, 13, 54-70.
 
Rieber, L. (1994). Computers, graphics, and learning. Madison: WCB Brown &

Benchmark.
 
Rieber, L. (1996). Seriously considering play: Designing interactive learning

environments based on the blending of microworlds, simulations, and games.
Educational Technology Research & Development, 44(2), 43-58.

 



Chen 63

Roblyer, M.D. & Knezek, G. (2003). New millenium research for educational
technology: A call for a national research agenda. Journal of Research on
Technology in Education, 36(1), 60-71.

 
Schofield, N. J. & Kirby, J. R. (1994). Position location on a topographical map:

Effects of abilities, preferences, and instruction. Cognition and Instruction, 12, 35-
60.

 
Schwartz, N.H. & Kulhavy, R.W. (1981). Map features and the recall of discourse.

Contemporary Educational Psychology, 6, 151-158.
 
Sweller, J. (1999). Instructional design in technical areas. Camberwell, Australia:

ACER.
 
Stanney, K.M. & Salvendry, G. (1995). Information visualisation: Assisting low

spatial individuals with information access tasks through the use of visual
mediators. Ergonomics, 38(6), 1184-1198.

 
Toh, S.C. (1998). Cognitive and motivational effects of two multimedia simulation

presentation modes on science learning. (Unpublished doctoral thesis). Centre for
Instructional Technology and Multimedia, University Science Malaysia.

 
Wilson, B.G. (1997). The postmodern paradigm. In R.D. Charles & A.J.

Romiszowski (Eds), Instructional development paradigms (pp. 297-309). New
Jersey: Educational Technology Publications.

 

 Willis, J. (2000). The maturing of constructivist instructional design: Some basic
principles that can guide practice. Educational Technology, 40(1), 5-16.

 

 Willis, J. & Wright, K. (2000). A general set of procedures for constructivist
instructional design: The new R2D2 model. Educational Technology, 40(2), 5-20.

 
Winn, W.D. (1993). A conceptual basis for educational applications of virtual reality.

Human Interface Technology Laboratory. (Tech. Rep. R-93-9). Seattle, WA:
Human Interface Technology Laboratory.

 
Youngblut, C. (1998). Educational uses of virtual reality technology. (Tech. Rep. IDA

Document D-2129). Alexandria, VA: Institute. [verified 1 Mar 2006]
http://www.hitl.washington.edu/scivw/youngblut-edvr/D2128.pdf

Dr Chwen Jen Chen, Faculty of Cognitive Sciences and Human
Development, Universiti Malaysia Sarawak. http://www.unimas.my/
Email: cjchen_usm@yahoo.com