Microsoft Word - [10]16-95-7-CE.doc Australasian Journal of Educational Technology, 2013, 29(3). ascilite 434 Getting inside the black box of technology integration in education: Teachers' stimulated recall of classroom observations Jo Tondeur, Ghent University (Research Foundation Flanders), Belgium L. H. Kershaw, Edith Cowan University, Australia R. Vanderlinde, and J. van Braak Ghent University, Belgium This study explored the black box of technology integration through the stimulated recall of teachers who showed proficiency in the use of technology to support teaching and learning. More particularly, the aim of the study was to examine how these teachers use technology in their lessons and to gain deeper insights into the multifaceted influences affecting their current practices. In order to explore this black box, observations and stimulated recall interviews with primary school teachers were conducted in schools which were selected by the inspectorate on the basis of advances they had made in educational technology use. Stimulated recall interviews – a verbal reporting technique in which the teachers were asked to verbalize their thoughts while looking at their own classroom practice on video – seemed to be a promising approach to increase authentic understandings of technology integration. The results emphasize that (a) the teachers involved in this study were pedagogically proficient and flexible enough to fit technology in with the varying demands of their educational practices, (b) the teachers' ongoing learning experiences rather than training affected the development of the quality of their practices, and (c) the role of the school and the broader context of teachers' personal lives played an important role. By interpreting the results of the study, recommendations are discussed for teacher technology integration and future research. Introduction Research in the classroom documents that teachers are beginning to integrate technology in their classrooms (e.g., Newhouse & Clarkson, 2008; Tondeur, Cooper, & Newhouse, 2010). According to these studies, the use of technology during teaching and learning activities is steadily increasing. But integrating technology is a complex process of educational change, and the use of technology in schools is still extremely varied (Lim, Zhao, Tondeur, Chai, & Tsai, 2013; Spector, 2010). Spector (2010) argues that with the potential of technology we can now do things to improve education that were not possible twenty years ago, but the potential gains in learning and instruction have yet to be realized on a significant global level. In this respect, Lei and Zhao (2007) demonstrate that the quantity of technology use alone is not critical to student learning. In their study, technology uses that had a positive impact on students were those related to specific subject areas and focused on student construction of knowledge. Although there is still much to learn with respect to the role and the impact of educational technology, the studies mentioned above indicate that some teachers are motivated to use the potential of technology in educational practice, while others do not share this affinity. To understand these reasons, many researchers have focused on critical characteristics associated with educational technology use, such as technical and human resources, as well as teachers' innovativeness, their computer experience and their beliefs about education (Hermans, Tondeur, van Braak, & Valcke, 2008; Prestridge, 2009, 2010). Moreover, recent studies indicate that technology integration depends on more than teacher-related characteristics. To illustrate, teachers in schools with an ICT school policy plan use technology more regularly in their classrooms (Tondeur, Van Keer, van Braak, & Valcke, 2008b). This confirms that successful technology use also depends on the development of a shared school vision. Australasian Journal of Educational Technology, 2013, 29(3). 435 While ample evidence now exists of factors which contribute to successful technology integration in education, which will be reviewed in more detail in the background section, many puzzling aspects still remain. It is this perspective of successful integration, rather than the exploration of impediments, which was the focus of the current study. The starting point of this research is that technology integration in education must be studied from a system of factors that interact in influencing the success of technology in education; the status of one factor is continuously affected by the status of many others (Sang, Valcke, van Braak, & Tondeur, 2010). This perspective was informed by researchers who have argued for a more holistic approach to encompass the dynamic connectedness of personal, pedagogical and organizational factors of influence on the integration of technology (Tondeur et al., 2008b; Way & Webb, 2007). The purpose of this study was to look at what is actually going on in the classroom, to collect more in- depth information on why teachers integrate technology in education and to describe the interrelated aspects contributing to their current practices. In order to achieve this aim, observations and stimulated recall interviews with primary school teachers were conducted. The data collection was restricted to six teachers from three primary schools in Flanders, the Dutch speaking region of Belgium. Stimulated recall is a verbal reporting technique in which the teacher is asked to verbalize his/her thoughts while looking at his/her own behaviour on video. In the results section, the profiles of primary school teachers are provided around three themes: (a) a portrait of selected teachers, (b) a description of observed lessons and (c) the rationale on why the teachers decided to use technology in that way. The similarities and differences among the cases are presented in the discussion section. Before presenting the results, it is useful to first elaborate upon the complexity of technology integration and the choice of stimulated recall as a method of data collection. This will be followed by a description of the design study and conclude with a discussion on the implications for practice. Background Integrated technology use: focus on teachers' classroom practice The current study focused on the questions of how and why teachers integrate technology in ways which showed a degree of proficiency in the support of teaching and learning. This focus was also adopted by other researchers investigating perspectives of successful technology integration (e.g., Northcote, Mildenhall, Marshall, & Swan, 2010; Trinidad, Newhouse, & Clarkson, 2004). These studies included descriptors about teacher capacities such as pedagogical knowledge of technology integrated planning, delivery of student-centred authentic learning experiences incorporating problem solving and higher order thinking skills, and knowledge about social and ethical issues. Teachers in such a technology setting have been described as those teachers who are innovators, flexible and motivated, have an understanding of students' learning and prior experiences, know about and employ a broad range of teaching strategies, and recognize the teaching opportunities and possibilities of technology tools and resources, and are committed to continuous learning (Cuban, 2002; Lopate, Miller, & Miller, 2003). The descriptions of teachers' capacities to integrate technology as mentioned above can also be found in the form of explanatory rubrics as progressive stages of teacher professional development (e.g., Kramarski & Michalsky, 2009). This study was concerned with different approaches to developing the skills of teachers, specifically the integration of technology and the associated pedagogical content knowledge within teachers' lesson planning and the monitoring of their growth through self-regulatory measures. Lessons were assessed using rubrics relating to elements of the learning phases of "planning, action and performance, and evaluation" (p. 481), which in turn had been affected by the different instructional approaches. The development of stages and levels of teacher capacities has assisted in illuminating the progress of teachers towards the integrated use of technology. For example, a teacher's pedagogic practice has been considered as emerging, applying, integrating or transforming (Jones, 2002) or at a level of entry, adoption, adaptation, appropriation or invention (Cuban, 2002; Dwyer, Ringstaff, & Haymore Sandholtz, 1994). Understanding quality pedagogy and good practice in a technology context was one of the aims of a comprehensive study undertaken in primary schools by Trinidad et al. (2004). They developed a technology framework consisting of three layers and five stages of teacher progress towards good practice. The latter two stages of integration and transformation provide descriptions of practice in terms Australasian Journal of Educational Technology, 2013, 29(3). 436 of teachers using technology as a tool and catalyst to support the learning, for example, in the attainment of multiple learning outcomes and supporting learners collaboratively. This research about teachers' progress in educational technology use, such as placing teachers' practices at various stages, levels, phases or within rubrics, by listing attribute descriptors about their capacities or enumerating influences on their characteristics, while significant, does not explain why teachers have reached such a degree of proficiency. Knowledge about the intricacies of actual classroom practices and the intimate nature of decision making in authentic settings are other facets which could shed further light on ways of expanding teacher growth and proficiency – an aim of all educators with a desire of improving the delivery of quality learning experiences for students. It was this aspect of authentic classroom practices from which this current research was developed. The educational practices in this study were cases in schools, selected by the inspectorate. The tasks of the Inspectorate in Flanders are "to check whether the schools comply with the regulations for recognition and financing or subsidising, whether the schools comply with the regulations concerning school-specific policy and whether the schools systematically monitor their own quality" (OECD, 2011, p. 82). The teachers in these schools were chosen by the principal, whose judgement is considered to be a reliable indicator of good practice (based on Lopate et al., 2003). Factors affecting technology use: getting inside the black box While the research literature as summarised above is comprehensive, it does not explain why teachers in schools provide experiences for their students within technology integrated learning activities. Despite the range of factors contributing to technology integration in the classroom, such as capacity of resources and sustainability of the infrastructure or teacher skills and attitudes (see Sang et al., 2010), these factors are insufficient to explain why the majority of teachers have not demonstrated successful technology integration. A possible reason is that it is not easy to fully understand this innovation solely on the basis of quantitative studies that often focus on the impact of individual and technology-related factors. Research has turned its attention to the multifaceted influences of teacher characteristics on practice (Tondeur, Van Keer, van Braak, Valcke, 2008; Way & Webb, 2007). To illustrate, in her quest to understand the relationships between pedagogy and practice, Senjov-Makohon (2006) explored the learning processes and attributes of teachers, their needs, abilities and learning strategies. She found that experienced teachers needed to engage actively in concrete experiences embedded in an ICT environment where collaborative and social interactions enabled them to overcome challenges and solve problems. In an in-depth study of a small group of teachers, pedagogic capacities to employ technology in innovative ways seemed to be affected by teachers' understandings of how technology could enhance learning outcomes (Hughes, 2005). In this study a series of professional development activities were conducted. Their focus was on pedagogic content knowledge and how to use technology in innovative and creative ways within a subject area. Findings indicated there were differences between novices and experienced teachers in this regard and it was suggested that collaborative, subject-specific learning experiences for all teachers would be beneficial. A change in pedagogic practice or growth in pedagogic knowledge to integrate technology within learning activities is implied as a desired outcome by the preceding examples. The latter aligns with the concept of technological, pedagogical and content knowledge (TPACK) provided by Koehler and Mishra (2009). TPACK has been created as a conceptual framework for the knowledge base teachers need to effectively teach with technology. This framework stems from the idea that technology integration in a specific educational context benefits from a careful alignment of content, pedagogy and the recognition of the opportunities which technology can provide (for an overview see Voogt, Fisser, Pareja Roblin, Tondeur, & Van Braak, 2013). Besides the importance of teachers' TPACK, it is also acknowledged that teachers' educational beliefs play an important role in the use of technology in the classroom (Hermans et al., 2008; Prestridge, 2009, 2010). Technological devices such as a computer or an interactive whiteboard do not embody one single pedagogical orientation; they include a spectrum of approaches to teaching and learning. According to Tondeur et al. (2008a), teachers select applications of technology in line with their selection of other curricular variables and processes that fit into their existing beliefs about "good" education. In other words, the role technology plays in education depends on choices about the nature of teaching and learning processes that teachers prefer and want to realize. Australasian Journal of Educational Technology, 2013, 29(3). 437 Apart from the factors at a teacher level, recent studies also reveal that technology is successfully integrated within educational activities because of particular school factors (Devolder, Vanderlinde, van Braak, & Tondeur, 2010; Trinidad et al., 2004). For instance, it appears that teachers in schools engaged in technology planning are more likely to apply technology in an innovative way (Tondeur et al., 2008b). Analysis of the available research also reveals the importance of leadership in managing technology integration effectively (e.g., Vanderlinde, Dexter, & van Braak, 2012; Vanderlinde, van Braak, & Tondeur, 2010). According to this study, such key personnel are in a position to create the conditions to develop a shared ICT policy within the school as a whole. Other school-related factors that can be connected to educational technology use are the degree of training and support (e.g., Devolder et al., 2010). Design of the study Purpose From this background it was noted that more in-depth information is needed on how and why teachers integrate technology within planned learning activities for their students. The approach used in the current study can be explained by the model depicted in Figure 1 (Hermans, 2009, based on Meijer, 1999). Figure 1. Latitudes and generalizability (Hermans, 2009, based on Meijer, 1999). In the field of research on educational technology use, the concept of latitude refers to the degree of freedom an individual teacher is given to make his or her thoughts about technology integration explicit. Generalizability refers to the possibility of making comparison between data from different teachers and the ability to generalize the findings to a population. The more the generalizability increases, the more the latitude inclines to the researcher (Figure 1). Since teacher characteristics like computer attitudes, innovativeness or educational beliefs cannot be observed in a direct way and they have to be inferred from peoples' actions and speech, it is hard to assess them in a straightforward way (cf. Hermans, 2009). Currently judgments about teacher characteristics in the use of technology, however, are made mostly on teachers' self-reported skills and pedagogical knowledge, with limited knowledge of the development and relationship between their acquired pedagogies and classroom practices (Cuban, 2002). Australasian Journal of Educational Technology, 2013, 29(3). 438 Recommendations from research studies in this field suggest that observations of teachers' practices would be a useful approach (e.g., Schepens, Aelterman, & Van Keer, 2008). The current study aimed at extending causal analytic approaches to understanding teachers' characteristics by using their stimulated recall of observed classroom practices where technology was integrated. The visual aids serve to elicit underlying beliefs in explaining teachers' behaviour. According to Fox-Turnbull (2011) the advantage of this approach is that stimulated recall data allows participants to explain their decision-making and that the use of multimedia sources has the benefit of helping them recall the technological practice undertaken. More specifically, this study explored the practices of primary school teachers in educational technology use, and described the interrelated aspects which have contributed to their current practices. The research method which follows explains more fully the approaches adopted in the investigative process. Participants The main aim of the present study was to explore (a) practices of teachers in their educational technology use and (b) the relational aspects of their progress toward this practice through the observations of lessons and stimulated recall interviews. The data collection was restricted to primary school teachers in Flanders, the Dutch speaking region of Belgium. As stated before, the teaching practices in this study were cases in three schools, selected by the Flemish inspectorate, irrespective of whether those practices exhibited features of a specific educational paradigm (cf. Law, Yuen, Ki, Li, Lee, & Chow, 2000). From the selected schools, we invited the three schools situated in Eastern Flanders, one of the five Flemish provinces. All three schools were willing to participate. The teachers in these schools were chosen by the principal. The literature shows that the judgements of advocates such as principals are reliable indicators of good practice (based on Lopate et al., 2003). In other words, no conceptual criteria were used for the case selection. Table 1 Background information of the sample teachers School 1 School 2 School 3 William Kathy Gerald Anna John Lisa Profile Age 53 years 30 years 34 years 26 years 56 years 51 years Teaching experience 31 years 10 years 14 years 5 years 35 years 30 years ICT experience 15 years 10 years 12 years 5 years 5 years 5 years Year level Grade 3 Grade 2 Grade 4 Grade 6 Grade 5 Grade 6 Three schools were involved in the study. Two teachers at each school agreed to participate, resulting in data from six teachers, three of whom were female. Teacher age varied from 26 to 56 years. Table 1 describes some background characteristics of the selected teachers. Procedure A stimulated recall interview took place with each teacher focusing on his/her teaching practice lesson, following the filming of each teacher throughout the 50 minute lesson. From different studies (e.g., Schepens et al., 2008), it became clear that a stimulated recall interview of a 50 min lesson is very time- consuming. Hence, each stimulated recall interview was restricted to (a) only those video-recorded moments during which technology was used and (b) the essential steps in the specific lesson (e.g., introduction, teacher demonstration, practice opportunities, evaluation). The video material of the teacher's lesson was played immediately afterwards using digital images on a portable computer. The actual stimulated recall interview lasted one hour to an hour and a half. During the stimulated recall interview the teachers stopped the video tape when they remembered what they were thinking while they were teaching. Also the interviewer could stop the tape in order to collect more in-depth information on Australasian Journal of Educational Technology, 2013, 29(3). 439 why teachers decided to use technology in that way in their lesson. Considering the validity of the stimulated recall procedure, a few questions were asked about the difficulty to relive the lesson after the interview (i.e., "Were you able to relive what you thought and felt during teaching by means of the video recordings?"). All teachers stated that they did not feel distracted by the camera and equally that they did not think that their students were distracted. Analysis During the actual stimulated recall interviews the interviewer's assistant coded teachers' utterances a first time. The purpose of this first coding, following the advice of Miles and Huberman (1994), was to explore the potential and possible biases in the coding procedure. In a next step, the two coders individually coded one interview from the same teacher. Based on a preliminary scanning, three broad categories were formed, namely, teacher characteristics, school characteristics, and types of technology use. In a next step of the coding, a technique of inductive thematic analysis was used. Inductive codes were assigned to segments of data that described a new theme observed in the text (Miles & Huberman, 1994). After the coders had completed this process, they compared the coded units to see how the units matched across the two coders' work and to discuss the meaning of each category in detail (cf. Schepens et al., 2008). All disagreements between the coders were resolved through discussion. In the next phase of analysis, the data from each teacher was brought together and a vertical analysis or within-case analysis was applied, synthesizing information for each case. This led to the creation of a case-specific report which organized and presented the interpretative data of the six teachers. In the second data analytic phase, the results of the within-case analysis were submitted to a cross-case analysis in which the data from the respondents were systematically compared for similarities and differences (Miles and Huberman, 1994). Also during this stage of data analysis, the content of cross-case analysis was discussed among the researchers to safeguard the quality of the interpretative data. Although presented as a linear, step-by-step procedure, the research analysis was an iterative process. Results Descriptive profiles of primary school teachers around three themes are provided (within-case analysis): (a) a portrait of selected teachers, (b) a description of an observed lesson and more specifically the practices of teachers and pupils that relate to technology and (c) the rationale on why teachers decided to use technology in that way. For reasons of space, in the next section only three of the six cases have been presented to illustrate key distinctions. Although the choice of the three teachers is subjective, it was decided to select one teacher from each school. Observations and initial data analysis suggest a number of common trends in technology use within the schools (Table 2). More specifically, teachers in the same school show a similar pattern of technology use such as the use of digital whiteboards for Math (School 1), the use of the computer lab to search and present information for project-based learning in the Dutch language course (School 2), and or the use of computers in the classroom to make mind maps in history (School 3). In the appendix a summary of the key findings of the six cases was made. The results of the horizontal analysis are integrated in the discussion section. William's case Portrait William, a third grade primary classroom teacher in his early fifties, had 31 years teaching experience and 15 years experience with computers. He was allowed one day per week to fulfil the role of the ICT technical coordinator, while a colleague had a similar position at the school for ICT curriculum integration. The school had a comprehensive ICT policy plan including clear goals with respect to the availability of ICT support and the provision of quality infrastructure. Two computer labs were scheduled for use by for classes and a website where free available software was listed for each grade level. Observed lesson: teaching symmetry using an interactive whiteboard (IWB) The observed Math lesson consisted of three parts. During the first part of the lesson William used a mirror to introduce the topic (symmetry) and to demonstrate the key concepts of the lesson (mirror image, Australasian Journal of Educational Technology, 2013, 29(3). 440 symmetry, etc.) with objects in the classroom. In the second part of the lesson, William used the IWB to look for images on the Internet and to verify whether the pupils could draw a symmetry line on these images on the IWB (individual responses to questions from teacher). In a more advanced exercise the pupils had to draw a mirror image (reverse graphical image) on the IWB of a figure made by the teacher. Therefore the software Symmetry was used. Finally, the pupils went to the computer lab so each pupil could practise independently, using the same software program. William's progress toward educational technology use From the stimulated recall interview, it became clear that William's progress in the use of educational technology was as a result of his personal interest. Although he didn't want to call himself a technical expert, he had gradually gained considerable technical knowledge and skills, through self-study and cooperation with colleagues. I remember that when I was a child I was already passionate in technology. That's part of me. I easily use new technologies just like it's a piece of myself. Formal professional development programs had played a minor role in his progress towards his current level of competency while colleagues in his school community had been a major influence. This could be illustrated by the introduction of the IWB in his school. Together with the principal and some colleagues, he visited another school where they observed and discussed the use of IWB in the classroom. After the decision to buy IWB's, he explored their possibilities for his own practice: The IWB arrived before summer holidays. During the holidays I've learned to use the IWB and searched for the added value of this tool in my class. William expressed the added value in the use of an IWB in terms of efficiency. As illustrated in the lesson reported above, he used the Internet on his IWB to "bring in the world into his classroom and to show how different kinds of objects could be mirrored". This meant, according to William, that he was no longer dependent on published educational resources, but that he could decide to expand this with images he had sourced and which were relevant to the lesson. This also made the lesson more attractive, but more importantly, according to William, was that a tool like an IWB could lead to the expansion of pupil's understandings: Using technology in the lessons is not only important because youngsters grow up in a digital age, it shows them how things we learn, such as "symmetry", can be applied in other aspects of life. Another reason why William used technology was for differentiation, as can be seen in the last part of the lesson where the pupils could work at their own level. He therefore used educational software for two purposes: "for those who are behind and need some more support, and those who are more advanced and need some challenging exercises. The advantage of technology is that I can easily differentiate". In addition to his personal motivation as a critical influence on his development of educational technology use, he acknowledged the importance of support at school level. Together with his colleague ICT coordinator, another colleague and the principal, William was part of the ICT team at his school. This team offered support and sustainability with respect to technology integration while encouraging him to "offer pupils sufficient opportunities to learn with technology in order to cope with the demands of the information society". Anna's case Portrait Anna as a 26-year old grade six teacher had five years professional experience. Together with her colleague Gerald, she was an "ICT contact person". The ICT coordinator was responsible for technical support in the school, while Anna offered didactical ICT support. The school had developed and documented a time line reflecting the ICT activities according to each grade level based on the ICT attainment targets. This document gave all teachers the opportunity to record and share their activities with other teachers thus making the progress of pupils' ICT use more transparent. These activities were linked to a website for the school community with relevant free software. With respect to hardware, Anna Australasian Journal of Educational Technology, 2013, 29(3). 441 had three PC's, a TV with video/DVD and a radio in her classroom with accessibility to a school projector and weekly use of the computer lab. Observed lesson: A project about books in the computer lab The observed lesson was part of a project about books. In this one-week project, the six graders worked together in groups of four on the same tasks about a different book. Tasks in which the pupils were engaged included creating a new ending, searching for information about the book and the author, promoting the book to other pupils and writing a review. This fourth lesson of the project took place in the computer lab where the group members worked individually or in smaller groups on the different tasks, such as making a digital story with a new ending of the book by using PowerPoint. Other pupils were looking for information on the Internet about the author and the book in order to make a brochure. There was a lot of mobility in the computer lab because the group members had to discuss and help each other. They were also communicating via email and using their memory stick to show their work to each other. Where necessary, Anna was coaching the different groups with relative little direction. During the first lesson(s) of the project I have more control in order to make sure the pupils understand the task. Next, they work in groups so they can help each other: some pupils know better how to look for information on the Internet while others are better in editing. Based on the technological expertise in the group they divide the work. In order to monitor the pupils' progress, the pupils used a logbook and had to email the output to Anna at the end of the lesson. Anna's progress toward educational technology use From the start of her career, Anna had integrated technology in her lessons. It was clear from the interview that she had a personal interest in technology before she became a teacher. I like to work with computers. This means that I look for appropriate ICT applications for my class and school on a daily base. Actually I spend a lot of evenings with my laptop. Anna's expertise had been built on self-study and the help of colleagues. She hadn't followed any specific training processes related to the acquisition of knowledge about technology and expected her pupils to do the same. As illustrated in the observed lesson she allowed pupils to first explore on their own how ICT applications work and then to ask for help from peers. According to Anna, collaboration with colleagues is the key to successful technology integration in education: "We exchange software, websites, and ideas about how to use technology. I even teach other teachers about specific applications. I don't do this because I'm the "contact person" for ICT, but because I believe teachers have to help each other." Communication was an essential part of technology usage. This was clearly illustrated during the observed lesson where pupils were sending emails to each other and to the teacher. Anna noted that in the ICT school-plan communication with ICT was a major stream for the third grade. Therefore she had started a mail project with a French speaking school in Belgium. Another example was that sometimes pupils had to send their homework via email to the teacher. It appeared from the interview that, although Anna was exploring a new program about how to chat simultaneously with different pupils, she was not intending to use social network sites in her teaching practice. I don't use like Facebook in my classroom because most of the pupils use this at home. What I do however is to discuss the possibilities and dangers of such applications. While looking at the video, Anna mentioned that there were insufficient computers for each child: "This [lack of infrastructure] was no problem in this specific lesson where pupils had to work together. For other subjects, such as Mathematics, pupils have to use math software to exercise their skills independent". Australasian Journal of Educational Technology, 2013, 29(3). 442 John's case Portrait John as a 56-year old teacher of grade five pupils had 35 years teaching experience, but had only been using technology in his educational practice for the last five years. During this period he had been ICT coordinator, responsible for the integration of technology in the curriculum. Last year, the principal had decided to appoint this coordination role to another teacher, but nevertheless John was still working closely with his ex-colleague, the (technical) ICT coordinator. The school has no ICT policy plan and technology-related matters, such as the purchase of laptops, are directed by the principal. In John's classroom there were five new PC's in addition to the five existing ones. John also used his own laptop to connect to a beamer, though shared with a colleague. Other equipment in his class was a digital camera and a TV with video. Finally, the school had a computer lab available with new laptops. Observed lesson: Learning history using digital mind maps During the introduction of the history lesson, John introduced a new theme - the Middle Ages by asking his pupils what they knew about this topic. After 10 minutes of brainstorming, he explained the main aim of the lesson which was to make a mind map based on (a) their existing knowledge about the Middle Ages and (b) the text in the handbook. He demonstrated a software program, eMindMaps, by making the first branches. The mind map was projected onto a screen in front of the classroom so the pupils could follow the instructions. In the next step, some pupils were invited individually to add new and related ideas to the same mind map. After the demonstration in the classroom, the pupils went to the computer lab to complete the mind map individually. Therefore, they received a printout of the mind map that was constructed but not finalized in the classroom. First, they had to give the file a name and save it as their personal map. Each pupil had a personal map on a virtual disk in order to save and change their work and to give John the opportunity to check the quality of the work. During this part in the computer lab, the ICT coordinator was helping the pupils with the more technical issues while John was walking around to help with content-related questions. For the last part of the lesson, the initial mind map was projected and the pupils had to help in completing the class mind map, based on their own work. During the last five to ten minutes the content of the mind map was discussed together with some reflections about the quality of the mind map, e.g., the difference between key ideas and sub ideas, where to insert a figure, the importance of using colours, etc. John's progress toward educational technology use John reported that his first technological experiences were influenced by his private life. It was his wife who bought a computer, his brother who bought him a mobile phone and thanks to his children John stayed in touch with the latest technologies. Because of the pressure of his family he started using technology, but gradually he became intrinsically motivated to use technology and to integrate it in his educational practice. Still, John mentioned a tension between the obligation to use technology and his personal motivation: I cannot buy everything such as an MP3-player or an IPad. It would be nice if I receive it, but no, at my age I give up in following the latest technologies. At the same time John realized that these new technologies could have a positive impact on his teaching practice: It's a new stimulus for me. I was using the same handbooks for almost 20 years and my lessons became routine. Technologies are an incentive to rethink my lessons. I would never use technology if I didn't need it for my profession. John participated in some courses in order to learn some technical skills to work with some basic programs such as a word processor whilst teaching himself some other skills. More importantly, Australasian Journal of Educational Technology, 2013, 29(3). 443 according to John, was the support of his ICT coordinator. To illustrate, the use of mind maps in the observed lesson was an idea of the ICT coordinator. Interestingly, the coordinator gives support during some of the ICT related lessons, as in the observed lesson. He [ICT coordinator] is a great motivator. He makes the machines work and brings changes. Actually, I am using what he's offering. With respect to infrastructure, John was happy with the five new computers in his classroom. "11 computers in my classroom is a good stimulus to integrate technology more often in my classroom practice". For John the challenge was how to use technology in an effective way. He argued that technology could make the lessons more attractive, but that it was no guarantee for better learning. Using ICT is like cooking: you first have to try it ten times and then you have to taste. It's not because you read a cookbook that you have a nice meal. While watching his teaching strategies in the videoed lesson, John noticed that using ICT was not always in line with his beliefs about education. For John discipline and structure were important. He felt that using technology could lead to chaos and fragmented content. "In my time everything was more structured: waiting in a row, you don't touch each other, we had to be quiet, etc. I've seen how all of this evaluated. Did you see in the beginning of the lesson how hard it was to get them quiet?" In most lessons, all pupils were using the same software as in the lesson presented above. He only allowed pupils to do something else on the computer when they had completed their work. His belief about the educational value of technology was that it facilitated classroom practice, but that it would not necessarily transform his teaching practice. According to John, textbooks were still the starting point for content delivery. He valued technology as a way of the pupils reinforcing their knowledge and practicing their skills. During the observed lesson for instance the pupils used their books to source information about the Middle Ages and the technology to make a mind map of the content. In order to match the right software he made use of a digital platform created by the ICT coordinator where the software was listed according to the grade levels and the specific subjects. Cross-case analysis and discussion This study builds on the knowledge and understandings of the complexities of the black box in the field of technology integration in education. The results about the reconstruction of the teachers' developmental progress in their use of technology in relation to their observed practices provided a variety of insights. These findings helped in understanding what critical factors had led to this level of proficiency in technology use (see Table 2). In the next section the cross-case results are discussed in three parts: (a) the teachers' developmental progress in educational technology use; (b) the role of the (school) context in this process, and (c) the teachers' approaches towards technology. This next section refers to the teachers (T) and schools (S) presented in Table 2. Teachers' developmental progress in educational technology use From the background section of this study it seemed that an important characteristic of teachers using technology includes their motivation and interest in exploring the opportunities and possibilities of technology for teaching and learning (Cuban, 2002; Lopate et al., 2003). All the teachers involved in this study experimented with technology applications, researched the use of technology in their education, gradually gained ideas for using technology in their subject lessons and shared their ideas and materials with colleagues. It appeared that the teachers' own learning experiences affected the quality of their practices, though training did not seem to be an important influencing factor (cf. Berliner, 2001). The latter refers to what Drent and Meelissen (2008) describe as personal entrepreneurs: "teachers who are willing to keep extensive contacts with colleagues and experts for the sake of his own professional development" (p. 197). The results of the case studies support the importance of personal entrepreneurship for the integration of technology in the learning process. Not surprisingly, five of the six Australasian Journal of Educational Technology, 2013, 29(3). 444 sample teachers were allowed some time per week to fulfil the role of technology coordinator in their school (see Table 2). Table 2 Summary of the cross-case analysis of the six sample teachers School 1 (S1) School 2 (S2) School 3 (S3) William (T1) Kathy (T2) Gerald (T3) Anna (T4) John (T5) Lisa (T6) Observed lesson Subject/ Content Math: symmetry Math: multiplication tables Dutch language / Math: clock reading Cross curricular project: week of youth novel History: Middle Ages Math: divisible by 4 and 25 Technology/ location/ Instructional strategy IWB for demonstration and didactic instruction in class + PC lab to practice skills (individual) IWB & audio player for demonstration and didactic instruction in class + PC lab to practice skills PC's & projector in PC lab for game based learning to practice skills (groups of 2: peer tutoring) PC's in lab to use ICT as Information tool (groups of 4) Laptops & projector to use ICT to make mind maps: in class + PC lab (individual) PC's in class as a learning tool to practice skills (individual) Critical factors Teacher & school factors lead to their practice Personal interest in ICT; allowed to be technical ICT coordinator; support at school level (e.g., ICT team, principal); collaboration with other schools, good infrastructure Allowed to be curriculum ICT coordinator; collaboration with other ICT coordinators; clear ICT school policies; good infrastructure Allowed to be "ICT contact person"; ICT school plan with time line; sharing activities with other teachers; website with free software Personal interest in technology; ICT school plan with time line; sharing activities with other teachers; help from pupils Influenced by his family; use of ICT to innovate; support from ICT coordinator; website with free software; use of own laptop; availability of 12 laptops 8 PC's in the class; website with free software; help from ICT coordinator Interesting to note is that the nature and degree of progress varied according to the different starting points and concerns of the selected teachers. For instance, John (T5/S3) was inspired by his private life to use technology only within the last five years. Kathy (T1/S1) started using technology when she was appointed by the school as technology coordinator. From these cases, it is clear that we have to take into account the context of the teachers. According to Berliner (2001), it is the power of context followed by deliberate practice which influences a teacher's level of competency. Livneh and Livneh (1999) have also taken a strong environmental position and claim that teachers need opportunities to reflect, engage in professional dialogue, peer observation, coaching and feedback to improve their educational practice. This corroborates the findings of this case study which identified a number of critical incidents for successful technology integration, such as internal and sustained support from the ICT coordinator, consultation with experts from outside the school, structuring dialogue between schools. Formal and informal learning at school level It is evident from the literature that besides traditional technology training courses, informal learning in school, e.g., from the technology coordinator or colleagues, and out of school, such as visits to other schools, should be considered (Tondeur, Devos, Van Houtte, van Braak, & Valcke, 2009). In this respect the results of the stimulated recall interviews stress the role of the school to provide "just in time support" or "support on demand" to encourage effective technology integration. Reflecting on the three sample schools, Schools 1 and 2 clearly articulated their goals and the means to support teachers to realize these goals in a technology plan. Research shows that creating such plans is a crucial step in the process of Australasian Journal of Educational Technology, 2013, 29(3). 445 technology integration (e.g., Vanderlinde et al., 2010). The technology plan of both schools incorporated a time line including the technology-related activities according to each grade level (S1/S2). By developing this time line, the sample teachers were not to be considered as recalcitrant recipients but as structurally constrained participants, highlighting the impact of communication in technology integration (cf. Tang & Ang, 2002). Interestingly, in two schools (S1/S2), the activities included in the technology plan were linked to a website with relevant free software (see also Pynoo et al., 2012). The two selected teachers from School 3 also made use of such an educational portal but this school did not have clear goals and support for technology integration. In this respect, Fullan (2006) argues that in an increasingly networked society such lateral capacity building is required where members of different schools learn from each other. Besides the availability of quality infrastructure and technical support in the different cases, the provision of curriculum support has been recognised as having a significant influence on technology integration (Tondeur et al., 2010). The findings in this study support this (see Table 2). Half of the sample teachers (T1/T2/T3) were leaders in the field of technological curriculum support and the other teachers were closely connected to the technological curriculum coordinator. In the context of this study, technological leadership had been clearly distributed in two of the three sample schools. To illustrate, the sample teachers of School 1 and 2 were members of the technology school team. William for instance, as a member of the team in School 1, was a technical coordinator, while his colleague Kathy had a similar role at the school for curriculum integration of technology. They were also working closely together with the principal and other teachers in order to realise successful technology integration. In the complex process of technology integration, the challenging tasks of leading schools are simply too great for responsibility to be invested in only one person. This is related to the field of distributed leadership, where leadership is an emergent property of a number of individuals who pool their expertise and work together in concerted way (see Vanderlinde et al., 2012). This means that a larger number of people are involved in technology integration, trusted with information, involved in decision making and participating in knowledge creation and transfer (cf. Sergiovanni, 2001). Approaches towards educational technology use What is clear from the exploration of the different cases is that the respondents were pedagogically highly proficient and flexible enough to fit technology in with the varying demands of their educational practices. More specifically, the observed lessons in combination with the recall interviews provided evidence that the teachers developed an integrative knowledge base of technological knowledge and skills, as well as knowledge of learners, subject matter content, and pedagogy necessary to teach with technology in the classroom. This has been described by Koehler and Mishra (2009) as TPACK. The results of this study indicate that the teachers were able to plan and monitor their TPACK (cf. Kramarski & Michalsky, 2009). The sample teachers also formed their own beliefs about the role of technology as a teaching tool (cf. Prestridge, 2010). In this respect Ertmer, Ottenbreit-Leftwich, and York (2007) refer to exemplary technology-using teachers as those who employ technology in learner-centred, constructivist environments as opposed to traditional teacher-directed environments. It has to be stated that the selection of the teachers in this study was not based on whether their practices exhibited features of a specific educational paradigm. As can be seen in the case lessons, technology was used for different reasons: for differentiation, as an information tool, as a tool to make mind maps, etc. (see Table 2). The results reveal that the type of technology use was related to the grade level, the specific subject content, and the school context, such as the availability of whiteboards (S1). Moss et al. (2007) caution that this approach may reinforce a relatively conservative use of technology as teachers adapt it to their existing practice. However, some of the sample teachers (T1/T4/T5) explicitly stated that technology also helped them to create more learner-centred environments. In the observed lesson of Anna for instance, technology was used as a tool and catalyst to support the learning in the attainment of multiple learning outcomes and in supporting learners' collaboration. Relevance of the findings and further research Although the present study has been helpful to explore in-depth the practices of teachers in their educational technology use and the relational aspects of their progress toward this practice, it also reflects Australasian Journal of Educational Technology, 2013, 29(3). 446 some shortcomings. As stated in the purpose section, the results of stimulated recall interviews have a limited generalizability. However, the added value lies in the deeper, richer account and the broader context, even though the results cannot simply be generalized to other educational levels, for instance. Some themes were specifically connected to the context of primary education. In secondary education for example, technology might be used in different ways. Furthermore, we have to assume that technology integration in education can be different outside the Flemish educational context. Apart from the evaluation of technology use at other educational levels and outside the Flemish context, there is also the question of how to select the sample teachers and the specific lessons. In future studies, researchers might wish to collect long-term in-depth information through observations and/or interviews in different parts of the world to help develop the findings of this study. Despite the limitations, the current study contributes to the literature on technology and education in different ways. This study illustrates that getting inside the black box of technology integration through a stimulated recall approach has the potential to increase authentic understandings of teachers' practices. The visual aids allow participants to explain their decision-making and the use of video sources has the benefit of helping them recall the practices undertaken. To illustrate the relevance of this approach, the findings of the stimulated recall interviews emphasise that technology became a tool in the ongoing development of the teacher's practice, rather than an instrument of imposed transformation. The teachers in this study were alert to, and welcomed change, provided it was in manageable steps. This finding has clear implications for professional development. As stated earlier, this developmental approach is supported by recent research for peer-working and networking as an efficient mode of continuing professional development. Next to the importance of personal commitment for technology integration in education, a critical implication for the school and the broader context can be derived from the stimulated recall interviews. From the recall interviews it became clear that most support is supplied in the school. It is therefore recommended that school leaders and ICT coordinators work closely together with the teachers to address their concerns about technology and provide an influential level of personal support and additional resources. It is in the workplace that opportunities for learning have to be provided and that positive reinforcement and support should be offered. As technology continues to drive changes in society and in education, we contend that local school policies need to define their organisational vision and actions more clearly in view of planned change. In a best-case scenario, these school policies stimulate the dialogue among school leaders, ICT coordinators, and teachers about the role of technology in education. Clearly, a supportive, blame-free environment that encourages and facilitates professional dialogue, and provides opportunities to extend and experiment with new practice can promote and further advance teaching and learning with technology. References Berliner, D. C. (2001). Learning about and learning from expert teachers. International Journal of Educational Research, 35(5), 463-468. Cuban, L. (2002). Oversold and underused: Computers in the classroom. Massachusetts: Harvard University Press. Devolder, A., Vanderlinde, R., van Braak, J., & Tondeur, J. (2010). Identifying multiple roles of ICT coordinators. Computers & Education, 55(4), 1651-1655. Drent, M. & Meelissen, M. (2008). Which factors obstruct or stimulate teacher educators to use ICT innovatively? Computers & Education, 51, 187-199. Dwyer, D. C., Ringstaff, C., & Haymore Sandholtz, J. (1994). The evolution of teachers’ instructional beliefs and practices in high-access-to-technology classrooms (Apple classrooms of tomorrow.. Report No. 8). Retrieved from http://www.apple.com/nl/images/pdf/acotlibrary/rpt8.pdf Ertmer, P., Ottenbreit-Leftwich, A., & York, C. (2007). Exemplary technology use: Teachers' perceptions of critical factors. Journal of Computing in Teacher Education, 23(2) 55-61. Australasian Journal of Educational Technology, 2013, 29(3). 447 Fox-Turnbull, W. (2011). Autophotography: A Means of stimulated recall for investigating technology education. In C. Benson & J. Lunt (Eds.), International Handbook of Primary Technology Education. International Technology Education Studies, 7, 195-210. Rotterdam: Sense. Fullan, M. (2006). Beyond turnaround schools. San Francisco: Jossey-Bass. Hermans, R. (2009). The influence of educational beliefs on the use of ICT as an educational innovation in primary education. (Unpublished doctoral dissertation). Ghent University, Ghent. Hermans, R., Tondeur, J., van Braak, J., & Valcke, M. (2008). The impact of primary school teachers' educational beliefs on classroom use of computers. Computers & Education, 51(4), 1499-1509. Hughes, J. (2005). The role of teaching knowledge and learning experiences in forming technology- integrated pedagogy. Journal of Technology and Teacher Education, 13(2), 277-302. Jones, A. C. (2002). The dichotomy of the conquering hero: Searching for the pedagogy in the teaching of ICT. Paper presented at the Concepts and Procedures. Proceedings of SITE 2002: Society for Information Technology & Teacher Education International Conference. Koehler, M. & Mishra, P. (2009). What is technological pedagogical content knowledge (TPACK)? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70. Kramarski, B., & Michalsky, T. (2009). Three metacognitive approaches to training pre-service teachers in different learning phases of technological pedagogical content knowledge. Educational Research and Evaluation, 15(5), 465-485. Law, N., Yuen, H. K., Ki, W. W., Li, S. C., Lee, Y., & Chow, Y. (Eds.). (2000). Changing classrooms and changing schools: a study of good practices in using ICT in Hong Kong schools. Hong Kong: CITE, University of Hong Kong Lei, J., & Zhao, Y. (2007). Technology uses and student achievement: A longitudinal study. Computers & Education, 49, 284-296. Lim, C. P., Zhao, Y., Tondeur, J., Chai, S. C., & Tsai, C. C. (2013). Bridging the gap: Technology trends and use of technology in schools. Journal of Educational Technology & Society, 16(2), 59-68. Livneh, C., & Livneh, H. (1999). Continuing professional education among educators: Predictors of participation in learning activities. Adult Education Quarterly, 49(2), 91–106. Lopate, C., Miller, K. A., & Miller, R. H. (2003). Survey of actual and preferred use of cooperative learning among exemplar teachers. The Journal of Educational Research, 96(4), 232-239. Meijer, P. C. (1999). Teachers' practical knowledge: Teaching reading comprehension in secondary education. (Unpublished doctoral dissertation). Leiden University, Leiden. Miles M. B. & Huberman A. M. (1994). Qualitative Data Analysis. Thousand Oaks, CA: Sage Publications. Moss, G., Jewitt, C., Levaaic, R., Armstrong, V., Cardini, A., & Castle, F. (2007). The interactive whiteboards, pedagogy and pupil performance evaluation: An evaluation of the Schools Whiteboard Expansion (SWE) Project: London Challenge. London: Department for Education and Skills. Newhouse, C. P., & Clarkson, B. D. (2008). Using learning environment attributes to evaluate the impact of ICT on learning in schools. Research and Practice in Technology Enhanced Learning, 3, 139-158. Northcote, M., Mildenhall, P., Marshall, L., & Swan, P. (2010). Interactive whiteboards: Interactive or just whiteboards? Australasian Journal of Educational Technology, 26(4), 494-510. Australasian Journal of Educational Technology, 2013, 29(3). 448 OECD (2011). School evaluation in the Flemish community of Belgium 2011 (OECD Reviews of Evaluation and Assessment in Education). OECD Publishing. Retrieved from http://dx.doi.org/10.1787/9789264116726-en Prestridge, S. (2009). Teachers' talk in professional development activity that supports change in their ICT pedagogical beliefs and practices. Teacher Development, 13(1), 43-55. Prestridge, S. (2010, April). The alignment of digital pedagogy to current teacher beliefs. Paper presented at the Australian Computers in Education Conference, Melbourne, Australia. Pynoo, B., Tondeur, J., van Braak, J., Duyck, W., Sijnave, B., & Duyck, P. (2012). Teachers' acceptance and use of an educational portal. Computers & Education, 58(4), 1308-1317. Sang, G., Valcke, M., van Braak, J., & Tondeur, J. (2010). Student teachers' thinking processes and ICT integration: Predictors of prospective teaching behaviors with educational technology. Computers & Education, 54, 103–112. Schepens, A., Aelterman, A., & Van Keer, H. (2007). Studying learning processes of student teachers with stimulated recall interviews through changes in interactive cognitions, Teaching and Teacher Education, 23(4), 457-472. Senjov-Makohon, N. (2006, December). How experienced teachers learn in the information age. Paper presented at the ASCILITE, Sydney, Australia. Retrieved from http://www.ascilite.org.au/conferences/sydney06/proceeding/pdf_papers/p201.pdf Sergiovanni, T. (2001). Leadership: what's in it for schools? London: RoutledgeFalmer. Spector, J. M. (2010). An overview of progress and problems in educational technology. Interactive Educational Multimedia, 1, 27-37. Tang, P. S., & Ang, P. H. (2002). The diffusion of information technology in Singapore schools: A process framework. New Media & Society, 4, 457–478. Tondeur, J., Cooper, M., & Newhouse, P. (2010). From ICT coordination to ICT integration: A longitudinal case study. Journal of Computer Supported Learning, 26(4), 494-506. Tondeur, J., Devos, G., Van Houtte, M., van Braak, J., & Valcke, M. (2009). Understanding organisational and cultural school characteristics in relation to educational change: The case of ICT integration. Educational Studies, 35(2), 223-235. Tondeur, J., Hermans, R., van Braak, J., & Valcke, M. (2008a). Exploring the link between teachers educational beliefs profiles and different types of computer use in the classroom: The impact of teacher beliefs. Computers in Human Behavior, 24(6), 2541-2553. Tondeur, J., Van Keer, H., van Braak, J., & Valcke, M. (2008b). ICT integration in the classroom: Challenging the potential of a school policy. Computers and Education. 51, 212-223. Trinidad, S., Newhouse, P., & Clarkson, B. (2004). A framework for leading school change in using ICT. In S. Trinidad & J. Pearson (Eds.), Using ICT in education: Effective leadership, change and models of best practice. Singapore: Pearson Education Asia. Vanderlinde, R., Dexter, S., & van Braak, J. (2012). School-based ICT policy plans in primary education: Elements, typologies, and underlying processes. British Journal of Educational Technology, 43, 505- 519. Vanderlinde, R., van Braak, J., & Tondeur, J. (2010). Using an online tool to support school-based ICT policy planning in primary education. Journal of Computer Assisted Learning, 26, 434-447. Australasian Journal of Educational Technology, 2013, 29(3). 449 Voogt, J., Fisser, P., Pareja Roblin, N., Tondeur, J., & Van Braak, J. (2013). Technological pedagogical content knowledge. A review of the literature. Journal of Computer Assisted Learning 29(2), 109-121. Way, J., & Webb, C. (2007). A framework for analysing ICT adoption in Australian primary schools. Australasian Journal of Educational Technology, 23(4), 559-582. Corresponding author: J. Tondeur, jo.tondeur@ugent.be Australasian Journal of Educational Technology © 2013. Please cite as: Tondeur, J., Kershaw, L. H., Vanderlinde, R., & van Braak, J. (2013). Getting inside the black box of technology integration in education: Teachers' stimulated recall of classroom observations. Australasian Journal of Educational Technology, 29(3), 434-449.