chang-cy.pdf


Australasian Journal of
Educational Technology

2012, 28(Special issue, 6), 983-999

MAGDAIRE: A model to foster pre-service teachers’
ability in integrating ICT and teaching in Taiwan

Chun-Yen Chang, Yu-Ta Chien, Yueh-Hsia Chang and Chen-Yung Lin
National Taiwan Normal University

This report describes our efforts in fostering Taiwanese pre-service teachers’ ability to
integrate information and communication technology (ICT) and teaching. The current
state of science teacher education and ICT policies in Taiwan are documented briefly.
The course model, MAGDAIRE (abbreviated from Modeled Analysis, Guided
Development, Articulated Implementation, and Reflected Evaluation) is then
introduced to counterbalance the insufficiency of the current teacher education courses
in Taiwan. The model is subsequently employed to enhance a science teacher
education course at National Taiwan Normal University (NTNU). The study
conducted along with the course indicates that MAGDAIRE can significantly improve
pre-service teachers’ ICT knowledge and skills. Moreover, MAGDAIRE can
significantly enhance pre-service teachers’ sensitivity to the complex interactions
between ICT, pedagogy, and subject matter. Some recommendations for employing the
proposed model to enhance teacher education courses are discussed.

Introduction

Educating all pre-service primary and secondary teachers in the use of information
and communication technology (ICT) to enhance teaching has become a social
commitment and an important objective for all teacher education programs in Taiwan.
In 2005, the Ministry of Education (MOE) in Taiwan enacted the project Establishing
ICT Literacy Standards for Primary and Secondary Teachers (MOE, 2008a). The project
argued that a modern teacher in this digital age must know how to utilise ICT for
enhancing instructional planning, teaching, assessment, and classroom management. It
was expected that, by the end of 2011, more than 90% of primary and secondary school
teachers would have implemented ICT-integrated curriculums into their classrooms
(MOE, 2008b). In order to meet this demand, the Primary and Secondary School
Information Education White Paper (MOE, 2008b) further emphasised the need to initiate
training programs to build up pre-service teachers’ ICT capacity for teaching.

However, there has been little pedagogical innovation to reframe ICT-related courses
for teacher education programs. The course list, issued by MOE for all teacher
education programs, has not been updated since 2004. As shown in Table 1, only one
ICT-related course can be found on the list of required courses; the course is called
Instructional Media, in the field of instructional methods. However, pre-service teachers
are asked to take only three courses out of the six courses in the field of instructional
methods. Thus this ICT course has to compete with other courses for survival within
the same field, including Principles of Teaching, Classroom Management, Educational
Assessment and Evaluation, Theory and Practice in Counseling, and C u r r i c u l u m



984 Australasian Journal of Educational Technology, 2012, 28(Special issue, 6)

Development and Design. In essence, ICT knowledge and skills are optional rather than
fundamental requirements for pre-service teachers. Besides, the focus of Instructional
Media is to train pre-service teachers in multiple ICT skills at a time, such as the use of
word processing, spreadsheet, the Internet, and presentation tools. As observed by
Angeli (2005), the role of ICT in this type of training is regarded as the content to be
learned, rather than the tool to enhance teaching. As a result, pre-service teachers may
lose sight of how ICT tools can be aligned with appropriate pedagogies for teaching.

Table 1: Education courses issued by the Ministry of Education in Taiwan

Field Courses Credit hours Remark
Introduction to Education 2
Educational Psychology 2
Sociology of Education 2

Foundations of
education

Philosophy of Education 2

Enrol in 2
courses

Principles of Teaching 2
Classroom Management 2
Educational Assessment and Evaluation 2
Theory and Practice in Counseling 2
Curriculum Development and Design 2

Instructional methods

Instructional Media 2

Enrol in 3
courses

Teaching Materials and Methods 2Teaching practicum
Practicum 2

Optional Around 20 courses 2 for each Enrol in 6-7
courses

The mastery of ICT knowledge and skills has been recognised as a significant and
fundamental factor contributing to the implementation of ICT-integrated teaching in
previous studies (e.g., Govender & Govender, 2009; Mahdizadeh, Biemans & Mulder,
2008; Sørebø, Halvari, Gulli & Kristiansen, 2009). However, in order to effectively
enhance students’ learning outcomes through ICT-integrated teaching, teachers need
to further understand the interaction between the use of ICT, teaching pedagogies
(Chang, 2003), students’ characteristics (Chang & Tsai, 2005), and features of subject
matter (Chien & Chang, 2012). In other words, any successful ICT-integrated teaching
requires a comprehensive understanding of the mutually reinforcing relationships
between technological knowledge, pedagogical knowledge, and content knowledge
(Angeli & Valanides, 2009; Chien, Chang, Yeh & Chang, 2012; Lee & Tsai, 2010; Jang &
Chen, 2010; Koehler, Mishra & Yahya, 2007; Mishra & Koehler, 2006; Niess, 2005).
Mishra and Koehler (2006) proposed the Technological Pedagogical and Content Knowledge
(TPCK or TPACK) framework to conceptualise such relationships. In order to
implement effective ICT-integrated teaching, teachers need to understand the dynamic
interplays between how a subject matter might be shaped by the application of ICT
(i.e., technological content knowledge), how teaching as well as learning might be
changed by the use of ICT (i.e., technological pedagogical knowledge), and how to
represent and communicate specific concepts and topics of a subject matter to students
(i.e., pedagogical content knowledge).

However, the three fundamental elements of ICT-integrated teaching (i.e., ICT,
pedagogy, and content) are fragmented and thus separated in different courses of the
current teacher education program in Taiwan. As indicated by Jang and Chen (2010),
pre-service teachers in Taiwan acquire content knowledge from discipline courses of



Chang, Chien, Chang and Lin 985

their colleges and gain pedagogical knowledge from education specialisation courses
offered by teacher education programs, respectively. They then, may patch up multiple
ICT knowledge and skills from the Instructional Media course. This approach is
irrelevant to the development of either pre-service teachers’ content knowledge or
pedagogical knowledge. Such an approach has been criticised for its failure to correlate
teachers’ ICT knowledge and skills with their teaching (Angeli, 2005; International
Society for Technology in Education, 2008; Jang, 2008; Jang & Chen, 2010; Koehler et
al., 2007; Niess, 2005; Organization for Economic Cooperation and Development, 2010;
Wilson, 2003). A pragmatic approach to build up pre-service teachers’ ICT capability
for teaching is to provide them with a meaningful context in which ICT can be
pedagogically situated in the teaching of subject matter (Angeli & Valanides, 2009;
Chien et al., 2012; Enochsson & Rizza, 2009; Jang & Chen, 2010; Koehler et al., 2007).

Therefore, we built a course model based on the learning technology by design approach
(Koehler et al., 2007; Mishra & Koehler, 2006) to enhance a science teacher education
course in Taiwan. As stated by Koehler et al. (2007; p. 744), learning technology by
design means that “teachers work collaboratively in small groups to develop
technology-rich solutions to authentic pedagogical problems, and accordingly, they
learn about technology and pedagogy by actually using and designing educational
technology to teach specific content.” Beyond involving pre-service teachers in
designing ICT-based lessons, we further engage pre-service teachers in implementing
and then revising their ICT-based lessons. This process moves pre-service teachers
from the roles of passive users of ICT into active designers, content providers, and
teaching practitioners of ICT-integrated lessons. By this means, the course is
contextualised with the rich interplays between ICT, pedagogy, and content. The aims
of the course model are to equip pre-service teachers with ICT knowledge and skills
and to facilitate their development of TPACK.

A model to enhance a pre-service science teacher education
course

In the course model, Adobe Flash is chosen as the tool to develop ICT-based lessons.
The reasons are as follows: (1) Flash can help teachers to compile static images into
dynamic animation forms through automatic procedures. Scientific phenomena, which
are too small, large, fast, or slow to be observed or experienced directly with the
unaided human senses, may be displayed in this way; (2) Flash allows teachers to add
interactive functions into animations to enable students to manipulate the
visualisations or models. It helps teachers to build tools to assist students in
visualising, sharing, and testing their hypotheses to explain scientific phenomena; (3)
Flash enables animations to record, retrieve, and exchange users’ information on the
Internet. It benefits teachers in collecting students’ data, in terms of learning processes
as well as assessments; (4) Flash-made content has reached 99% of Internet viewers
(Adobe, 2010). Students and teachers can easily access any Flash-based resources both
inside and outside of schools. The aforementioned features of Flash allow pre-service
science teachers to customise ICT-based lessons to fit their teaching practices.

The use of Flash to build and implement the ICT-base lessons is simply one example of
pre-service teachers’ ICT competency. In this paper, the development of the pre-service
teachers’ pedagogical reasoning on the integration of Flash and teaching is thus
specified as the development of “Technological Pedagogical and Content Knowledge – Flash
(TPACK-F)”. Based on our previous study (Chien et al., 2012), we draw on Collins’s



986 Australasian Journal of Educational Technology, 2012, 28(Special issue, 6)

(1988) cognitive apprenticeship to scale up a 4-phase cyclic MAGDAIRE model
(abbreviated from Modeled Analysis, Guided Development, Articulated
Implementation, and Reflected Evaluation). The use of cognitive apprenticeship aims
to facilitate pre-service teachers to learn conditions for applying TPACK-F, see the
implications of TPACK-F, and make inventions and apply their TPACK-F to capture
the general features of teaching involved in classroom contexts, including complexity,
uncertainty, instability, uniqueness, and value conflict (Clark & Yinger, 1987). The
framework of MAGDAIRE can be seen in Figure 1; each phase of MAGDAIRE is
described below in more detail.

Figure 1: MAGDAIRE model (revised from Chien et al., 2012)

Phase I: Modeled analysis

Pre-service teachers are divided into sub-groups. Teacher trainers demonstrate ample
Flash-based online courseware of their own design. They then explain the design
thinking underlying their courseware, for peers to observe and imitate. In this phase,
the modeling facet of Collins’s cognitive apprenticeship is applied to facilitate their



Chang, Chien, Chang and Lin 987

comprehension of TPACK-F. The modeling of instructors’ performances and design
thinking aims to enable pre-service teachers to brainstorm what features of Flash
would be pedagogically powerful to present and support the chosen content; and how
Flash may be integrated into classroom teaching. Each group then selects a science
topic; they then enact their Flash-based online courseware blueprints for teaching the
science topic.

Phase II: Guided development

Teacher trainers introduce the Flash concepts and skills required for constructing a
Flash-based online courseware to pre-service teachers. Each group then tries to
transform the learning materials of the chosen subject matter into a Flash-integrated
format. The next step is to devise relevant activities as well as assessments to
complement the pedagogies they plan out. Thereafter, each group tunes the learning
materials, activities, and assessments into an online package. In this phase, the
coaching and scaffolding facets of Collins’s cognitive apprenticeship are applied to
facilitate pre-service teachers to transform TPACK-F into artifacts to solve authentic
pedagogical problems. Teacher trainers provide pre-service teachers with critical aids
in transforming their topic, learning activities, and assessments into a Flash-integrated
format. Teacher trainers further suggest or provide solutions to the problems pre-
service teachers encounter. As the pre-service teachers gain more expertise, teacher
trainers gradually push them to think and work more independently.

Phase III: Articulated implementation

Pre-service teachers are asked to present and execute their Flash-based online
courseware along with their instructional plans in a classroom setting. In this phase,
the articulation facet of Collins’s cognitive apprenticeship is applied to facilitate pre-
service teachers to articulate their tacit TPACK-F in teaching practice. Teacher trainers
induce pre-service teachers to explain their intentions and reasons for every step they
perform. This process forces the pre-service teachers to think about what they are
doing on Flash-integrated teaching. Thus, they must make their tacit thoughts
explicitly clear. This also enables them to experience others’ perspectives on Flash-
integrated instruction in the same context and across different contexts.

Phase IV: Reflected evaluation

Pre-service teachers are asked to compare their own performances with those of peers
and teacher trainers, and then give comments on others. In this phase, the reflection
and exploration facets of Collins’s cognitive apprenticeship are applied to facilitate
pre-service teachers to modify their TPACK-F. Each pre-service teacher reflectively
researches what elements are critical to successful and unsuccessful Flash-integrated
teaching. Moreover, the comments from peers function as a replay for analysis.
Instructors further encourage pre-service teachers to form hypotheses for refining their
Flash-based online coursewares and instructional plans. The reflected evaluation phase
takes place as the formative assessment. It prompts pre-service teachers to deliberate
and experiment. They are obliged to share the manner in which the subject matter and
teaching/learning processes might be shaped by the application of Flash in the next
cycle of MAGDAIRE.



988 Australasian Journal of Educational Technology, 2012, 28(Special issue, 6)

While the context of MAGDAIRE predetermines the production and implementation
of Flash-based online courseware as the solution to the pre-service teachers’
instructional problems, it allows them to explore various pedagogies with respect to
their subject matters. Our previous study in a similar context (Chien et al., 2012) found
that pre-service science teachers held different views of teaching when they entered the
context of MAGDAIRE. These different instructional orientations were reflected in
their Flash-based lessons, such as transmission-orientation and the constructivist-
orientation. Through inspecting peers’ performances on Flash-based lessons within the
context of MAGDAIRE, the pre-service teachers revised their courseware and
instructional plans to improve the efficiency of their Flash-integrated teaching
practices. It was observed that, in order to negotiate the ideal Flash-integrated
instruction with actual classroom settings, the pre-service science teachers’ Flash-
integrated pedagogies transited between transmission-orientation and constructivist-
orientation, such as linear learning pacing versus nested learning pacing, non-
interactive activities versus interactive activities, and structured lecturing versus open
inquiry. We believe that MAGDAIRE facilitates pre-service teachers’ investigation into
ICT-integrated pedagogies. It may be utilised to scaffold pre-service teachers in
exploring appropriate pedagogies.

A case study to evaluate the effectiveness of MAGDAIRE

Context of the study

MAGDAIRE was employed to run an 18-week long science teacher education course at
National Taiwan Normal University (NTNU). NTNU has served as the backbone
output source of secondary teachers for Taiwan since 1967 (MOE, 2006). The course
was worth 2 credit hours in the secondary teacher education program at NTNU. The
panel of teacher trainers consisted of a senior professor with expertise in science
education and computer-assisted instruction, a professional designer with expertise in
Flash-based courseware development, and a research assistant who holds a science
teaching license with experience in Flash-based courseware development. A total of
sixteen college students were enrolled in the course. These pre-service teachers had
science subjects as their majors, but none of them had a background in computer
science. The pre-service teachers voluntarily formed eight groups (two people per
group). Each group chose one science topic to develop Flash-based online courseware;
each group’s topic was different from that of the other groups. The topics included
wind, solar and lunar eclipses, cloud, global warming, tide, tsunami, typhoon, and
rock. The class met weekly for two hours in a computer lab to complete their projects.
The course content is shown in Table 2.

Research questions

The research questions, within the context of MAGDAIRE, guiding the current study
were:

RQ1: Are pre-service teachers’ proficiency in Flash knowledge and skills improved
after the course?

RQ2: Does the pre-service teachers’ TPACK-F transit towards a more connected model
that addresses Flash, pedagogy, and content jointly over time?



Chang, Chien, Chang and Lin 989

Table 2: Course content

Phase of MAGDAIRE Week Assignment and activity Datacollection
Phase I:

Modeled
analysis

1st
|

2nd

Pre-service teachers are divided into sub-groups
and observe an ampleness of online courseware
that teacher trainers demonstrate.
Teacher trainers lead pre-service teachers in
discussing the design of the demonstrated online
courseware; they engage pre-service teachers in
brainstorming their own online courseware
blueprints.

Proficiency
of Flash
knowledge
test (PFKT)
and
Proficiency
of Flash skill
test (PFST)
(week 1)

Phase II:
Guided

development

3rd
|

9th

Teacher trainers introduce the online courseware
design concepts and skills, including:
‧． Flash: Layer and animation
‧． Flash: Interactive functions
‧． Flash: Multimedia integration and playback

Teacher trainers scaffold pre-service teachers to
apply these skills to fulfill their own online
courseware blueprints. At the same time, pre-
service teachers devise relevant activities and
assessments to complement their own online
courseware.

Group
journal
(week 9)

Phase III:
Articulated

implementation

10th Pre-service teachers demonstrate their online
courseware prototypes with instructional plans
and explain their design thinking.

Group
journal
(week 10)

First
round

Phase IV:
Reflected

evaluation

11th Teacher trainers ask pre-service teachers to
evaluate and comment on other groups’
performances and give feedback.

Phase I:
Modeled
analysis

12th
|

13th

Teacher trainers lead pre-service teachers in
discussing how to strengthen the design of online
courseware.

Phase II:
Guided

development

14th
|

16th

Pre-service teachers collaboratively test the ideas
for  improving online courseware.
Teacher trainers encourage skilled pre-service
teachers to do peer coaching.

Group
journal
(week 16)

Phase III:
Articulated

implementation

17th Pre-service teachers demonstrate their online
courseware prototypes with instructional plans
and explain their design thinking.

Group
journal
(week 17)

Second
round

Phase IV:
Reflected

evaluation

18th Teacher trainers ask pre-service teachers to
evaluate and comment on other groups’
performances and give feedback.

PFKT and
PFST (week
18)

Data collection and analysis

In order to answer RQ1, the Proficiency of Flash Knowledge Test (PFKT) and Proficiency of
Flash Skill Test (PFST) were employed. PFKT consisted of 32 true-false items (a total of
32 points). PFST was a practical Flash design task (a total of 10 points). PFKT and PFST
were validated by our previous study (Chien et al., 2012), which includes the content
and scoring guidelines of the two tests. As specified in Table 2, PFKT and PFST were
administrated to the pre-service teachers at the beginning and end of the semester in a
pre-post test format. The Kudere-Richardson Formula 20 (KR20) coefficient was used to
determine the reliability of data collected by PFKT. A KR20 of .6 is generally considered
an acceptable level (Nunnally, 1978); the estimated KR20 was .9 and .6 for pre-PFKT and



990 Australasian Journal of Educational Technology, 2012, 28(Special issue, 6)

post-PFKT, respectively. A two-tailed t-test statistical analysis was conducted on the
mean differences between the pre-service teachers’ pre- and post-test scores of PFKT
and PFST to examine whether MAGDAIRE improved their proficiency of Flash
knowledge and skills. The statistical significance testing was conducted at the alpha =
.05 significance level by using Statistical Package for Social Sciences version 15.0.

To answer RQ2, this study traced the development of the pre-service teachers’ TPACK-
F by collecting and analysing their group journals at different phases of MAGDAIRE.
The main themes of the group journal were “What are the problems your group may
encounter when you implement your courseware in a classroom for teaching? How do
you plan to resolve the stated problem(s)?” As suggested by Koehler et al. (2007),
teachers’ ideas to solve design problems which they encounter in integrating ICT and
teaching reflect the components that dominate their pedagogical reasoning during the
process. Therefore, the record of teachers’ ideas may function as a proxy to identify the
forms of their TPACK. Koehler et al. (2007) analysed the records of teachers’
discussions on design problems they encountered when developing a website for
online courses, as a way to represent the teachers’ development of TPACK.

The current study used group journal writing as an alternative means to induce the
pre-service teachers’ ideas to solve design problems. As specified in Table 2, each
group of the pre-service teachers was engaged in writing group journals at the end of
Phase II (first round, week 9), Phase III (first round, week 10), Phase II (second round,
week 16), and Phase III (second round, week 17). Thus, a total of 32 group journals (4
phases each for 8 groups) was obtained. Conceptual analysis, which focuses on
establishing the existence and frequency of concepts in a text (Berelson, 1971), was
applied to categorise the types of design thinking within the pre-service teachers’
group journals. The number of occurrences was then computed. The process of
conceptual analysis in this study was conducted in the following three steps:

1. The level of analysis on each group’s journals was set as “sets of sentences
describing a solution to one problem regarding the implementation of group
projects.” The group journals were carefully reviewed by two researchers and then
divided into segments. A total of 128 segments were obtained from the group
journals.

2. We developed a TPACK-F coding protocol based on Koehler et al.’s TPACK work
(2007) and then coded the segments. The TPACK-F coding protocol can be seen in
Table 3. Two researchers used the protocol to analyse each segment and categorise
each segment into Content (C), Pedagogy (P), Flash (F), Content pedagogy (CP),
Flash content (FC), Flash pedagogy (FP), and Flash content pedagogy (FCP). The
inter-rater reliability was estimated by kappa coefficient. The kappa for the two
coders was .80, and the disagreement between the two coders was solved through
discussion. The data was re-coded by the same coders two weeks later for
enhancing coding reliability. The kappa among the two coding intervals was .84.

3. A Pearson chi-squared test was conducted to examine whether there was any
significant difference in the patterns of pre-service teachers’ pedagogical reasoning
on the integration of Flash and teaching in different phases of MAGDAIRE. The
statistical significance testing was conducted at the alpha = .05 significance level.



Chang, Chien, Chang and Lin 991

Table 3: TPACK-F coding protocol

Category
(code) Description Exemplar

Content
(C)

Reasoning on the actual subject
matter that is to be taught, such as
clarifying the facts, concepts, and
theories of the chosen subject
matter.

We have to clarify the explanation for the
greenhouse effect. When solar radiation passes
through the earth’s atmosphere, it warms the
planetary surface. The greenhouse gases in the
atmosphere absorb the infrared thermal radiation
emitted from the planetary surface. Furthermore,
the greenhouse effect already existed before the
occurrence of the so-called global warming.

Pedagogy
(P)

Reasoning on the processes and
methods of teaching and learning;
also, how it encompasses overall
educational purposes, values and
aims, such as arranging students’
learning steps.

We should administer a test to students by the
end of the course. It can help us to understand
students’ learning progress and thus offer us
information to give students appropriate
feedback.

Flash
(F)

Reasoning on the use of Flash but
not specifically related to the
chosen subject matter or teaching
strategies, such as the operation of
one particular function of Flash.

The animation may suddenly break off while
playing. Maybe we should use the frame-by-
frame approach to compile the animation.

Content
pedagogy
(CP)

Reasoning on how particular
aspects of a subject matter are
organised, adapted, and
represented for instruction, such
as specifying one teaching
strategy to complement one
particular concept of the chosen
subject matter.

By comparing with other subjects in the domain
of earth science, the learning unit of rocks puts
more emphasis on students memorising the facts.
In addition to introducing the characteristics of
varied kinds of rocks and Moh's Hardness scale,
we should encourage students to compare the
differences in the hardness, crystal system, crystal
class, and streak between minerals to enhance
their learning motivation.

Flash
content
(FC)

Reasoning on how the chosen
subject matter might be shaped by
the application of Flash, such as
leveraging one particular function
of Flash to present the chosen
subject matter.

The pictures of clouds can be embedded with
more detailed information by utilising Action
Script. We attempt to revise the scripts to make
the cloud characteristics, such as cloud
classification and cloud height which will appear
when the cursor is moved onto the cloud pictures.

Flash
pedagogy
(FP)

Reasoning on how teaching as
well as learning might be changed
by the use of Flash, such as
leveraging one particular function
of Flash to support one particular
teaching strategy.

The online testing system should be added, with
Action Script, to count students’ scores. If a
student types in wrong answers to a question too
many times, the system will automatically force
him/her to view the animation which explains
the concept of that question.

Flash
content
pedagogy
(FCP)

Reasoning on how the chosen
subject matter might be shaped by
the application of Flash and the
impact of such applications on
teaching methods, such as
leveraging one particular function
of Flash to present the chosen
subject matter and support one
particular teaching strategy.

We plan to add interactive functions into the
animation depicting sea waves, to allow students
manipulate the variations in the water depth
along the coastline. By this means, students can
test their hypotheses of the relation between
water depth and wave speed, and then discuss
the data they obtain from the animation with
peers. It will facilitate students in exploring the
relationship between water depth and wave
speed. It also helps to explain the difference
between deep and shallow waves to students.



992 Australasian Journal of Educational Technology, 2012, 28(Special issue, 6)

Findings and discussion

Advancement in proficiency of Flash knowledge and skills
As shown in Table 4, the results of two-tailed paired t-test analysis reveal that there are
statistically significant differences in their pre-post scores on both PFKT and PFST (p <
.001 for PFKT and p < .001 for PFST). The effect sizes computed from the t-test analysis
on the pre-service teachers’ pre-post test scores are beyond the large magnitude (d = 2.2
for PFKT and d = 1.6 for PFST). Moreover, by examining the pre-service teachers’ Flash-
based online courseware collected at the end of the course, we found that all of their
Flash-based online courseware can be operated by users on personal computers
without any functional errors. The screenshots of three samples of the pre-service
teachers’ finished products are presented in Figure 2. The quality of the pre-service
teachers’ courseware suggests that the pre-service teachers acquire commendable
technical knowledge and skills of Flash through the MAGDAIRE model. These
proficiencies enabled them to customise applicable Flash-based online courseware.
Overall, in a practical sense, MAGDAIRE can significantly enhance pre-service
teachers’ Flash knowledge and skills.

Table 4: Comparisons between pre-service teachers’ pre-test and post-test scores
Pre-test scores Post-test scores Two-tailed paired t-testTest n M SD n M SD t p d

PFKT 16 13.7 4.8 16 22.6 4.1 8.64 .000* 2.2
PFST 16 4.3 1.1 16 7.2 1.7 6.20 .000* 1.6

Figure 2: Screenshots of pre-service teachers’ Flash-based online science courseware

Development in TPACK-F
Table 5 presents the crosstabs statistics of the coded segments by phases. As shown in
Table 5, the data distribution violates the assumption of the chi-squared test because
there are 19 cells with frequency lower than 5, below the expected value. Therefore, we
combine the predefined categories to facilitate the computation of a chi-squared test.
The categories C, P, and F are combined into a new category called isolated concept,
whereas the categories CP, FC, FP, and FCP are combined into another new category
called integrated concept. As shown in Table 6, the results of the chi-squared test on the
combined categories indicate that there is statistically significant difference in the
patterns of pre-service teacher’ reasoning in the four phases (χ2[3] = 31.68, p < .001,
Cramer's V = 0.5, large effect size); and no cell has expected count lower than five. We
further examine the standard residuals in Table 6 to determine the cells with significant
differences. The significance level of post-hoc analysis was adjusted to .0083 (.05/6) by
applying Bonferroni correction. The results reveal that the occurrence of isolated
concepts in the pre-service teachers’ reasoning on design problems (i.e., C, P, or F) is
significantly higher than the expected value (p < .0083) at the end of Phase II (first



Chang, Chien, Chang and Lin 993

round). The occurrence of integrated concepts (i.e., CP, FC, FP, or FCP) is significantly
lower than the expected value (p  < .0083) at the end of Phase II (round 1). No
significant differences can be found in Phase III (round 1), Phase II (round two), and
Phase III (round 2).

Table 5: Crosstabs statistics of the distribution of coded segments over phases

Phase
Concept Phase II

(round 1)
Phase III
(round 1)

Phase II
(round 2)

Phase III
(round 2)

Total

Count 6 (18.2%) (a) 3 (8.6%) 2 (7.1%) 0 (.0%) 11
Expected count 2.8 3.0 2.4 2.8 11.0
Residual 3.2 .0 -.4 -2.8

C

Std. residual 1.9 .0 -.3 -1.7
Count 6 (18.2%) 2 (5.7%) 2 (7.1%) 0 (.0%) 10
Expected count 2.6 2.7 2.2 2.5 10.0
Residual 3.4 -.7 -.2 -2.5

P

Std. residual 2.1 -.4 -.1 -1.6
Count 13 (39.4%) 4 (11.4%) 0 (.0%) 8 (25.0%) 25
Expected count 6.4 6.8 5.5 6.3 25.0
Residual 6.6 -2.8 -5.5 1.8

F

Std. residual 2.6 -1.1 -2.3 .7
Count 4 (12.1%) 4 (11.4%) 2 (7.1%) 1 (3.1%) 11
Expected count 2.8 3.0 2.4 2.8 11.0
Residual 1.2 1.0 -.4 -1.8

CP

Std. residual .7 .6 -.3 -1.1
Count 2 (6.1%) 8 (22.9%) 2 (7.1%) 4 (12.5%) 16
Expected count 4.1 4.4 3.5 4.0 16.0
Residual -2.1 3.6 -1.5 .0

FC

Std. residual -1.0 1.7 -.8 .0
Count 2 (6.1%) 10 (28.6%) 14 (50.0%) 10 (31.3%) 36
Expected count 9.3 9.8 7.9 9.0 36.0
Residual -7.3 .2 6.1 1.0

FP

Std. Residual -2.4 .0 2.2 .3
Count 0 (.0%) 4 (11.4%) 6 (21.4%) 9 (28.1%) 19
Expected count 4.9 5.2 4.2 4.8 19.0
Residual -4.9 -1.2 1.8 4.3

FCP

Std. residual -2.2 -.5 .9 2.0
Count 33 (100.0%) 35 (100.0%) 28 (100.0%) 32 (100.0%) 128Total
Expected count 33.0 35.0 28.0 32.0 128.0

a. % within phase.

Overall, the results indicate that the pre-service teachers’ TPACK-F showed profound
changes towards a more connected model by the end of Phase III (round 1); by
comparing with Phase II (round 1), the percentage of the summation of integrated
concepts (i.e., CP, FC, FP, and FCP) raised from 24.2% to 74.3% at the end of Phase III
(round 1). Moreover, such connected model of TPACK-F was sustained in the pre-
service teachers’ reasoning at the end of Phase II (round 2) and Phase III (round 2); the
percentages of the summation of integrated concepts were relatively high (85.7% for
Phase II (round 2) and 75.0% for Phase III (round 2).



994 Australasian Journal of Educational Technology, 2012, 28(Special issue, 6)

Table 6: Crosstabs statistics of the distribution of combined categories over phases
Phase

Concept Phase II
(round 1)

Phase III
(round 1)

Phase II
(round 2)

Phase III
(round 2)

Total

Count 25 (75.8%) (a) 9 (25.7%) 4 (14.3%) 8 (25.0%) 52
Expected count 11.9 12.6 10.1 11.5 52.0
Residual 13.1 -3.6 -6.1 -3.5

Isolated

Std. residual 3.8* -1.0 -1.9 -1.0
Count 8 (24.2%) 26 (74.3%) 24 (85.7%) 24 (75.0%) 76
Expected count 21.1 22.4 17.9 20.5 76.0
Residual -13.1 3.6 6.1 3.5

Integrated

Std. residual -2.9* .8 1.4 .8
Count 33 (100.0%) 35 (100.0%) 28 (100.0%) 32 (100.0%) 128Total
Expected count 33.0 35.0 28.0 32.0 128.0

a. % within phase; * p < .0083

The results suggest that only engaging pre-service teacher in designing Flash-based
lessons is not sufficient to develop their TPACK-F. Before Phase III (round 1), the
TPACK-F was determined by the teacher trainers and was delivered to the pre-service
teachers. The pre-service teachers were expected to learn this knowledge and apply it
to build Flash-based courseware and design their Flash-integrated instructional plans.
The major role of the pre-service teachers was that of the passive receiver of TPACK-F.
In this stage, the pre-service teachers mainly treated content, pedagogy, and Flash as
isolated elements of Flash-integrated teaching; the coded segments of C, P, and F
occupied the pre-service teachers’ group journals (a total of 75.8%). Especially, the
Flash-only reasoning (F) dominated the pre-service teachers’ design thinking.

However, one week later (i.e., at the end of Phase III (round 1)), there was a significant
shift in the presence of Flash-related reasoning in the group journals. Flash was
addressed jointly with pedagogy (FP = 28.6%) or with content (FC = 22.9%), rather
than being thought of in the isolated form (F = 11.4%). Moreover, Flash became
integrated with both content and pedagogy at the end of this phase (FCP = 11.4%). The
plausible reason is that Phase III (round 1) forced each pre-service teacher to consider
what elements were critical to successful and unsuccessful Flash-integrated teaching. In
this stage, the pre-service teachers were not only receivers of TPACK-F but also
generators of TPACK-F. This process made the pre-service teachers rethink why and
how they used the functionalities of Flash to accomplish pedagogical goals by
comparing their own performance with that of others. Thus it facilitated the pre-
service teachers in re-examining the connection between the affordances of Flash and
teaching to revise their solutions to design problems. By examining the content of the
coded segments, it is found that at the end of Phase III (round 1), the pre-service
teachers tended to use Flash to complement the solutions related to content and/or
pedagogy issues which they previously proposed in Phase II (round 1).

An example of the transition from P to FP can be found in Ken’s group. Ken’s group
stated one concern for learning evaluation at the end of Phase II (round 1), “We should
add some tests into the curriculum. Only by this means, we are able to understand
students’ learning progress and provide them with aids in conquering learning
difficulties” (coded as P). Ken’s group then tended to leverage the Action Script of Flash
to enhance the process of learning evaluation at the end of Phase III (round 1), “we can
incorporate the automatic response mechanism into our tests. It not only helps us to



Chang, Chien, Chang and Lin 995

evaluate students’ learning progress but also promotes the interactivity of the
evaluation – it may enhance students’ motivation” (coded as FP). Another case of
transition from C to FC is that Tim’s group clarified their teaching focus on the
movement of winds at the end of Phase II (round 1), “We should emphasise some
situations in which the movement of winds is affected by Coriolis force. Upper-air
winds blow parallel to the isobars and reflect a balance between the pressure-gradient
force and the Coriolis effect” (coded as C); Tim’s group decided to draw on the
incorporation of Action Script and animations to better visualise the chosen subject
matter at the end of Phase III (round 1), “we plan to enable the wind direction,
depicted in the animation, to be manipulated by typing in pressure values and
switching the existence of Coriolis force” (coded as FC).

A case of the transition from FC to FCP from Albert’s group concerned the strategy for
teaching tsunami at the end of Phase II (round 1), “Students may encounter great
difficulties in recognising the differences between tsunami waves and normal sea
waves. We have to clarify the difference between these two types of waves in class”
(codes as CP). They then considered using animations to enhance the pedagogical
technique for teaching tsunami at the end of Phase III (round 1), “We want to use an
animation to present the profiles of both tsunami and normal sea waves. It helps
students to compare the differences between tsunami and normal sea waves, in terms
of both period and speed” (coded as FCP).

The aforementioned trend is also found across Phase II (round 2) and Phase III (round
2). In addition, the pre-service teachers tended to deliberate the niches of Flash-based
resources to empower their pedagogies (i.e., the transition from FC to FCP) at the end
of Phase II (round 2) and Phase III (round 2). An example of this may be seen when
Tim’s group decided to reshape their animation about the movement of winds at the
end of Phase III (round 1), “…to enable the wind direction to be manipulated by
typing in pressure values and switching the existence of Coriolis force” (coded as FC).
Tim’s group then relied on this animation to support students’ inquiry at the end of
Phase III (round 2), “…by manipulating the pressure-gradient force, Coriolis force, and
friction force in the animation, students are able to test their hypotheses about the
wind movement and demonstrate their experiments to peers and the teacher” (coded
as FCP). However, the pre-service teachers rarely identified the impacts of the Flash-
integrated pedagogies on their subject matter (i.e., the transition from FP to FCP). The
plausible reason is that the Flash-integrated pedagogies, which the pre-service teachers
proposed, are inherently not constrained by the characteristics of their subject matter.
For instance, the incorporation of the automatic response mechanism into assessments
is, in general, applicable to any subject matter.

An interesting phenomenon is found that a peak of Flash-only reasoning emerges from
Phase III (round 2) (F = 25%). By examining the content of the segments coded as F
across four phases, we found that the Flash issues, which concerned the pre-service
teachers in Phase III (round 2), were more advanced Flash subjects than those of the
other three phases. The F codes of Phase III (round 2) mainly focused on the
programming of Action Script, as one group stated at the end of Phase III (round 2),
“The chess of the losers in the game should be removed away from the chessboard. We
should program Action Script to hide the chess of the players who are eliminated
through competition” (coded as F). Those of the previous phases focused on animation
compilation, “…using the frame-by-frame approach to compile the animation” (coded
as F). The finding reveals that, by the end of MAGDAIRE, the pre-service teachers
tended to enhance their courseware with advanced Flash knowledge and skills. This



996 Australasian Journal of Educational Technology, 2012, 28(Special issue, 6)

implies that the pre-service teachers’ Flash knowledge and skills were promoted to a
higher level. In addition, it also illuminates the importance of the technical supports to
sustain pre-service teachers’ inquiry into advanced Flash knowledge and skills for
complementing teaching practices after the course.

Limitations

Some concerns about the power of statistical significance testing may be raised because
the sample size involved in this study is rather small. As a matter of fact, a small
sample size is inherently more difficult to achieve statistical significance than a large
one (Daniel, 1998; McLean & Ernest, 1998). However, the effect sizes reported in this
study all attain a large magnitude. It suggests that the statistically significant
differences reported in this study have practical significance. Moreover, the findings
will be much easier to obtain statistical significance and be replicated in a larger
sample size. It also should be noted that the claims drawn from the study could be
weakened if the coding scheme was flawed or the researchers made misjudgments
when coding the qualitative data. However, the coding scheme was cautiously
developed from a previous research (Koehler et al., 2007). We also attempted to
minimise subjectivity and enhance reliability while coding the collected data. We
suggest future research to conduct relational analyses on the interaction between ICT,
pedagogy, and content in teachers’ reasoning. Such analyses will be very valuable to
the field of TPACK research. The current study reports possible pathways for isolated
concepts of TPACK-F to evolve into integrated concepts of TPACK-F. These pathways
may be helpful to build a framework for future studies to conduct relational analyses
on teachers’ development of TPACK. However, additional data should be collected to
confirm speculations on these pathways. For example, interviews with teachers must
be conducted, to clarify the relationships between the ideas which they wrote down in
group journals during different phases of MAGDAIRE. This is one of the tasks planned
for while implementing MAGDAIRE in the next round in 2012.

Conclusion and recommendations
The findings of the current study echo with our pilot study (Chien et al., 2012) that
MAGDAIRE can significantly enhance pre-service teachers’ proficiency in Flash
knowledge and skills. Moreover, the current study further indicates that MAGDAIRE
can facilitate pre-service teachers’ TPACK-F development towards a more connected
model that addresses Flash, pedagogy, and content jointly. Some may doubt the
findings of the current study because it seems quite natural for pre-service teachers to
see a better connection between Flash, pedagogy, and content as they learn more and
have more experience with Flash. Though this presumption is rational, it is not
supported by previous studies. For instance, Koehler et al. (2007) used the TPACK
framework to analyse the discussion during a design seminar in which faculty
members worked in conjunction with masters students to develop online courses. The
analysis on the design-talk of two groups (i.e., the Adams Family group and the
Jackrabbits group) showed divergent patterns from a similar design-based problem
solving experience. When dealing with design problems within a semester-long
period, the Adams Family group showed profound changes towards a more connected
model, in which ICT was jointly considered with content and/or pedagogy. On the
other hand, the Jackrabbits group did not show such changes. Koehler et al. (2007)
concluded that the learning technology by design approach can enable the growth of
TPACK to occur, but did not guarantee it. However, this study into the MAGDAIRE
model shows a general growth of the pre-service teachers’ TPACK-F across groups.



Chang, Chien, Chang and Lin 997

The plausible reason is that MAGDAIRE strengthens and complements the original
version of the learning by design approach. The facets of cognitive apprenticeship put
in place in MAGDAIRE may further facilitate pre-service teachers to articulate and
modify their TPACK-F, and thus can foster pre-service teachers’ better development of
TPACK-F. Especially, engaging pre-service teachers in co-constructing TPACK-F is
found to be a critical mechanism for deepening their recognition of the interplays
between Flash, pedagogy, and content.

Further suggested potential accompaniments when employing MAGDAIRE to
enhance teacher education courses are as follows. Firstly, encourage pre-service
teachers to record their design thinking. We acknowledge that asking the pre-service
teachers to collaboratively turn in their group journals may function as an intervention
of the reported study. This strategy may facilitate pre-service teachers to become aware
of why and how they choose solutions to pedagogical problems. Moreover, this
strategy seems to have positive impacts on pre-service teachers’ reasoning between
ICT, content, and pedagogy. The solutions that the pre-service teachers described in
their group journals reflected more deepened reasoning over time in the study.

Secondly, we also advocate supporting pre-service teachers’ inquiry into ICT both
during and after the course. Pre-service teachers certainly will run into technical
problems that they find difficult to solve independently after classes, due to the limited
time of weekly formal courses. Thus there is a need to provide pre-service teachers
with online platforms to discuss with experts and peers what they experience
individually. The cases solved through the collaboration between experts and pre-
service teachers can be recorded online, so that they can be a reference for future
research. As revealed by the reported study, pre-service teachers tend to explore more
advanced ICT skills to complement their teaching practices (i.e., the programming
related to Action Script) by the end of MGADAIRE. Therefore, sustained technical
support is vital to encourage pre-service teachers to explore the pedagogical rationale
of ICT use. We have currently assigned a research assistant as a part-time facilitator to
help the pre-service teachers to overcome Flash-related issues even after they have
passed the course.

Acknowledgments
The authors wish to thank the Aim for the Top University (ATU) project of the
National Taiwan Normal University (NTNU) for financing this study. The authors
gratefully acknowledge the assistance of Chun-Chieh Huang and Terrence Wong. The
authors also thank anonymous reviewers for valuable comments.

References
Adobe (2010). Flash player statistics. [viewed 20 Nov 2010].

http://www.adobe.com/products/flashruntimes/statistics.html

Angeli, C. (2005). Transforming a teacher education method course through technology: Effects
on preservice teachers' technology competency. Computers & Education, 45(4), 383-398.
http://dx.doi.org/10.1016/j.compedu.2004.06.002

Angeli, C. & Valanides, N. (2009). Epistemological and methodological issues for the
conceptualization, development, and assessment of ICT–TPCK: Advances in technological
pedagogical content knowledge (TPCK). Computers & Education, 52(1), 154-168.
http://dx.doi.org/10.1016/j.compedu.2008.07.006



998 Australasian Journal of Educational Technology, 2012, 28(Special issue, 6)

Berelson, B. (1971). Content analysis in communication research. New York: Hafner Publishing
Company.

Chang, C. Y. (2003). Teaching earth sciences: Should we implement teacher-directed or student-
controlled CAI in the secondary classroom? International Journal of Science Education, 25(4),
427-438. http://dx.doi.org/10.1080/09500690210145701

Chang, C. Y. & Tsai, C. C. (2005). The interplay between different forms of CAI and students'
preferences of learning environment in the secondary science class. Science Education, 89(5),
707-724. http://dx.doi.org/10.1002/sce.20072

Chien, Y. T. & Chang, C. Y. (2012). Comparison of different instructional multimedia designs for
improving student science-process skill learning. Journal of Science Education and Technology,
21(1), 106-113.

Chien, Y. T., Chang, C. Y., Yeh, T. K. & Chang, K. E. (2012). Engaging pre-service science teachers
to act as active designers of technology integration: A MAGDAIRE framework. Teaching and
Teacher Education, 28(4), 578-588. http://dx.doi.org/10.1016/j.tate.2011.12.005

Clark, C. M. & Yinger, R. J. (1987). Teacher planning. In D. C. Berliner & B. Rosenshine (Eds.),
Talks to teachers. New York: Lane Akers, Inc.

Collins, A. (1988). Cognitive apprenticeship and instructional technology: Technical report. Cambridge,
MA: Bolt Beranek and Newman. http://www.eric.ed.gov/ERICWebPortal/contentdelivery/
servlet/ERICServlet?accno=ED331465

Daniel, L. G. (1998). Statistical significance testing: A historical overview of misuse and
misinterpretation with implication for the editorial policies of educational journals. Research
in the Schools, 5(2), 23-32. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.122.4807

Enochsson, A. & Rizza, C. (2009). ICT in initial teacher training: Research review. OECD Publishing.
http://www.oecd-ilibrary.org/education/ict-in-initial-teacher-training-research-
review_220502872611

Govender, D. & Govender, I. (2009). The relationship between information and communications
technology (ICT) integration and teachers’ self-efficacy beliefs about ICT. Education As
Change, 13(1), 153-165. http://dx.doi.org/10.1080/16823200902943346

International Society for Technology in Education (2008). National educational technology standards
and performance indicators for teachers. Eugene, OR: International Society for Technology in
Education. http://www.iste.org/standards/nets-for-teachers.aspx

Jang, S. J. (2008). The effects of integrating technology, observation and writing into a teacher
education method course. Computers & Education, 50(3), 853-865.
http://dx.doi.org/10.1016/j.compedu.2006.09.002

Jang, S. J. & Chen, K.-C. (2010). From PCK to TPACK: Developing a transformative model for
pre-service science teachers. Journal of Science Education and Technology, 19(6), 553-564.
http://dx.doi.org/10.1007/s10956-010-9222-y

Koehler, M. J., Mishra, P. & Yahya, K. (2007). Tracing the development of teacher knowledge in a
design seminar: Integrating content, pedagogy and technology. Computers & Education, 49(3),
740-762. http://dx.doi.org/10.1016/j.compedu.2005.11.012

Lee, M. H. & Tsai, C. C. (2010). Exploring teachers' perceived self efficacy and technological
pedagogical content knowledge with respect to educational use of the World Wide Web.
Instructional Science, 38(1), 1-21. http://dx.doi.org/10.1007/s11251-008-9075-4

Mahdizadeh, H., Biemans, H. & Mulder, M. (2008). Determining factors of the use of e-learning
environments by university teachers. Computers & Education, 51(1), 142-154.
http://dx.doi.org/10.1016/j.compedu.2007.04.004



Chang, Chien, Chang and Lin 999

McLean, J. E. & Ernest, J. M. (1998). The role of statistical significance testing in educational
research. Research in the Schools, 5(2), 15-22.

Mishra, P. & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework
for teacher knowledge. Teachers College Record, 108(6), 1017-1054.
http://www.tcrecord.org/Content.asp?ContentID=12516

MOE (2006). Important teacher cultivation policy. Taipei, Taiwan: MOE.

MOE (2008a). Information education report. Taipei, Taiwan: MOE.

MOE (2008b). Primary and secondary school information education white paper. Taipei, Taiwan: MOE.

Niess, M. L. (2005). Preparing teachers to teach science and mathematics with technology:
Developing a technology pedagogical content knowledge. Teaching and Teacher Education, 21,
509-253. http://www.citejournal.org/vol8/iss4/mathematics/article1.cfm

Nunnally, J. C. (1978). Psychometric theory. New York: Oxford University Press.

Organization for Economic Cooperation and Development (2010). Inspired by technology, driven by
pedagogy: A systemic approach to technology-based school innovations. Organization for Economic
Cooperation and Development Publishing. http://www.oecd.org/edu/ceri/inspiredby
technologydrivenbypedagogyasystemicapproachtotechnology-basedschoolinnovations.htm

Sørebø, Ø., Halvari, H., Gulli, V. F. & Kristiansen, R. (2009). The role of self-determination theory
in explaining teachers’ motivation to continue to use e-learning technology. Computers &
Education, 53(4), 1177-1187. http://dx.doi.org/10.1016/j.compedu.2009.06.001

Wilson, E. K. (2003). Preservice secondary social studies teachers and technology integration:
What do they think and do in their field experiences. Journal of Computing in Teacher
Education, 20(1), 29-39.

Authors: Dr Chun-Yen Chang (corresponding author), Professor and Director
Graduate Institute of Science Education, National Taiwan Normal University, Taiwan
Science Education Center, National Taiwan Normal University, Taiwan
Department of Earth Sciences, National Taiwan Normal University, Taiwan
Email: changcy@ntnu.edu.tw Website: http://w1.ceels.org/chunyen/

Yu-Ta Chien, PhD student, Graduate Institute of Science Education
National Taiwan Normal University, Taiwan. Email: yutachien@ntnu.edu.tw

Dr Yueh-Hsia Chang, Assistant Research Fellow, Science Education Center
National Taiwan Normal University, Taiwan. Email: yueh@ntnu.edu.tw

Dr Chen-Yung Lin, Professor and Dean
Graduate Institute of Science Education, National Taiwan Normal University, Taiwan
Office of Teacher Education and Careers Service, National Taiwan Normal University
Department of Life Science, National Taiwan Normal University, Taiwan
Email: lcy@ntnu.edu.tw

Chun-Yen Chang and Yu-Ta Chien contributed equally to this work.

Please cite as: Chang, C.-Y., Chien, Y.-T., Chang, Y.-H. & Lin, C.-Y. (2012). MAGDAIRE:
A model to foster pre-service teachers’ ability in integrating ICT and teaching in
Taiwan. In C. P. Lim & C. S. Chai (Eds), Building the ICT capacity of the next
generation of teachers in Asia. Australasian Journal of Educational Technology, 28(Special
issue, 6), 983-999. http://www.ascilite.org.au/ajet/ajet28/chang-cy.html