Australasian Journal of Educational Technology, 2015, 31(6). 699 Designing and evaluating a professional development programme for basic technology integration in English as a foreign language (EFL) classrooms Muhammad Fauzan Ansyari UIN Sultan Syarif Kasim Riau, Indonesia This study aims to develop and evaluate a professional development programme for technology integration in an Indonesian university’s English language teaching setting. The study explored the characteristics of this programme to English lecturers’ technological pedagogical content knowledge (TPCK) development. This design-based research employed interviews, a data logbook, TPACK (Total PACKage) surveys, teacher attitudes toward computer (TAC) surveys, and a technology integration assessment rubric. Results show that participants had positive experiences with the professional development programme. TPCK was also enhanced after the professional development activities based on self-reported TPACK survey. Overall, evidence from all data sources shows that the professional development arrangement for technology integration improves the English lecturers’ TPCK. Crucial aspects of a professional development programme should include the TPACK framework as a knowledge base, the design approach, active engagement, authentic learning experiences in a collaborative environment, curriculum coherency, an intensive programme schedule, guidance, support, and feedback. Introduction In the digital age, teacher competency in educational uses of technology appears to be one of the key components in some educational reform efforts (Pineida, 2011). It is believed that such competency can improve the quality of teaching and student learning outcomes as a response to the need for true learning in the 21st century, which entails students’ ability to use information and communication technology (ICT) for better learning (Jimoyiannis, 2010). In the context of English language teaching (ELT), although the impacts of technology on student outcomes are not univocal, appropriate and effective technology integration could make language learning more authentic and meaningful for students (Warschauer & Kern, 2000; Warschauer & Meskill, 2000; Young, 2003). Training events on technology have become a major part of educational programmes probably because lecturers find technology integration difficult. According to several authors (Haydn & Barton, 2008; Hew & Brush, 2007; Tondeur, van Keer, van Braak, & Valcke, 2008), a lack of knowledge and skills is one barrier to technology integration. Furthermore, Harris and Hofer (2009) assert that planning an appropriate and pedagogically powerful use of educational technologies into instruction is demanding. This is because such integration cannot make use of technology without considering content and pedagogy to facilitate learning. This indicates a need for an effective professional development programme to cope with such a problem. The results of the study presented in this paper aims to developing and evaluating a professional development programme for technology integration at the university level. This structure of this article begins with the aim of the study and the research question. It is followed by a literature review with the intention of establishing a theoretical basis for developing a professional development for technology integration in this study. It then details the design of the programme as the intervention in this study, research methodology, and finally research findings of the study. Aim of the study and research question In response to the need of ICT in education, UIN SUSKA Riau undertook some improvement strategies. The Information Technology (IT) Strategic Planning released in 2007 was part of its development plan. One of the main targets was to improve lecturers’ understanding and help them integrate technology into instruction (Universitas Islam Negeri Sultan Syarif Kasim Riau, 2007). However at the university’s Australasian Journal of Educational Technology, 2015, 31(6). 700 language centre, this policy has hardly been implemented to support ELT as a student-centred learning practice. On the basis of context analysis, language centre lecturers lack knowledge and skills for technology integration. As a result, students are not well supported to engage in authentic language uses and have few opportunities to practice the language. In some cases, the lecturers simply teach English without any technology support to enhance students’ learning. When the lecturers teach listening, for example, many do not use any relevant technology which is crucial to expose students to authentic spoken language by native speakers of English. Considering this reality, the researcher argues that there are some contributing factors that create barriers for technology integration, one of which is the lecturers’ lack of knowledge and skills in designing and using technology. The study described in this paper was therefore aimed at developing and evaluating a professional development programme for technology integration using the TPACK framework for English lecturers at UIN SUSKA’s Language Centre. The primary question of this study was: What are the characteristics of an effective lecturer professional development programme for technology integration in ELT at UIN SUSKA’s language centre? This main question was divided into the following sub-questions: 1. What do the English lecturers experience as they participate in the professional development programme for technology integration? 2. Has the lecturers’ TPACK improved after the professional development programme? Literature review Framework for teacher knowledge base in technology integration The problem with TPACK as a framework is that it has no universally accepted conceptualisation. This is indicated by the three conceptualisations of TPACK found in the literature. The first conceptualisation from Mishra and Koehler (2006) centres on TPACK as teachers’ understanding of the integrated knowledge domains of technological, pedagogical, and content knowledge in specific contexts. The second conceptualisation, the ICT-TPACK developed by Angeli and Valanides (2009), views TPACK as consisting of separate knowledge domains that can be developed and measured in isolation from one another. Finally, an elaborated TPACK, as conceptualised by Cox and Graham (2009), is simply an expansion of the original TPACK framework. In this view, TPACK refers to “the knowledge of how to coordinate the use of subject-specific or topic-specific activities with topic-specific representations using emerging technologies to facilitate student learning” (Cox & Graham, 2009; p. 64). The professional development programme described in this study used Koehler and Mishra’s model (2008) as the teacher knowledge base for technology integration. TPACK includes three knowledge domains, which consists of technological knowledge (TK), pedagogical knowledge (PK), and content knowledge (CK). Further, all these three bodies of knowledge intersect to form pedagogical content knowledge (PCK), technological content knowledge (TCK), and technological pedagogical knowledge (TPK). This framework is shown in Figure 1 below. PCK refers to how teachers teach particular content-based material to students. TCK is how teachers select and then use technologies to teach particular content knowledge, while TPK mainly addresses how teachers use particular technologies when they are teaching a certain subject matter. Finally, according to Koehler and Mishra (2005b; 2006), the intersection of the three circles forms a combination of the three knowledge domains, referred to as TPCK in this study (Thompson & Mishra, 2007). Australasian Journal of Educational Technology, 2015, 31(6). 701 Figure 1. TPACK framework and its knowledge components (www.tpack.org) Learning technology by design approach The learning-technology-by-design approach (Mishra & Koehler, 2006) is a constructivist model for teacher learning. Koehler and Mishra (2005a) recommend it as an effective approach with which teachers, content experts, and technology specialists, design instruction collaboratively, building TPACK together. As described by Mishra and Koehler (2006), this approach emphasises learning by design and doing on the part of practitioners. Since the design process is crucial, Mishra and Koehler (2006) argue that design should be understood as dialogue and action, and it involves reflection to provide learners with authentic and engaging ill-structured problems that reflect the complexity of the real world. Characteristics of successful TPACK professional development arrangements A number of critical considerations that should be taken into account when designing TPACK professional development programmes have been found in previous studies. Those characteristics include participants’ active involvement in the programme (Blocher, Armfield, Sujo–Montes, Tucker, & Willis, 2011; Jimoyiannis, Tsiotakis, & Roussinos, 2011), providing participants with authentic learning experiences (Doering, Veletsianos, Scharber, & Miller, 2009; Jimoyiannis, 2010; Tee & Lee, 2011), collaboration among participants (Allan, Erickson, Brookhouse, & Johnson, 2010; Jimoyiannis et al., 2011), providing guidance and support to participants when needed (Blocher et al., 2011; Doering et al., 2009; Guzey & Roehrig, 2009; Oster-Levinz & Klieger, 2010; Trautmann & MaKinster, 2010), having curriculum coherency with the programme (Nicholas & Ng, 2012), reflecting on what has been done (Allan et al., 2010; Guzey & Roehrig, 2009; Harris & Hofer, 2011; Jimoyiannis, 2010; Tee & Lee, 2011), giving feedback on participants’ works (Jimoyiannis, 2010; Niess, Zee, & Gillow-Wiles, 2010), providing intensive training (Trautmann & MaKinster, 2010), and having sufficient time for participants to learn and practice (Jimoyiannis, 2010; Nicholas & Ng, 2012; Oster-Levinz & Klieger, 2010). Studies have also revealed that participants had positive experiences in TPACK professional development programmes with such characteristics. Blocher et al. (2011) claimed that half of their participating teachers reported improvement in their comfort and confidence in using technology in the programme. Another study showed that all teachers reported increased willingness and confidence, particularly in their ability to apply ICT in their own instruction (Jimoyiannis, 2010; Jimoyiannis et al., 2011). Furthermore, Doering et al. (2009) asserted that in-service teachers who went through a TPACK-based programme experienced considerable movement within the TPACK diagrammatic knowledge domains and expressed positive and encouraging comments regarding their knowledge domains portrayed within the TPACK framework. http://www.tpack.org/ Australasian Journal of Educational Technology, 2015, 31(6). 702 Finally, teachers reported being satisfied with the programme and perceived that the programme had a positive impact on their development (Jimoyiannis et al., 2011). Design of the intervention In this study, both context analysis and literature review provided useful insights for the design of an effective TPACK professional development arrangement. Initial design guideline Contextual analysis and literature review of successful TPACK initiatives provided several key design guidelines to develop the intervention, as follows. 1. Technology integration into teaching requires a clearly articulated framework, providing teachers with clear understanding of how to integrate technology. Therefore, the TPACK framework as suggested by Mishra and Koehler (2006) is deemed appropriate for describing the knowledge domains needed by the English lecturers for technology integration in teaching English as a foreign language. 2. Koehler and Mishra’s learning technology by design approach (2005a) can be used to put the English lecturers in the role of curriculum designers. This means that lecturers work in a collaborative environment with a small group to develop solutions to their real instructional problems. 3. English lecturers should be engaged in their professional development programmes in order to become the main players and experience authentic learning in a collaborative environment (Mishra & Koehler, 2006). The programme should include workshop, course design, enactment of the designed course, and finally reflection and revision with sufficient time. 4. English lecturers should receive practical and technical support during the programme to enhance learning (Blocher et al., 2011; Doering et al., 2009; Guzey & Roehrig, 2009; Oster-Levinz & Klieger, 2010; Trautmann & MaKinster, 2010). 5. Programme content should be based on the curriculum (Nicholas & Ng, 2012). Development of the professional development programme The professional development programme for technology integration as the intervention in this study was designed to last 3 weeks in order to strengthen the learning and performance of lecturers concerning technology integration. It was also integrated with the English lecturers’ teaching activities so as to improve their current teaching practices, and organised under the language centre administration. Four phases were prepared to provide the English lecturers with authentic learning experiences during the professional development workshops: introduction, lesson plan design, implementation, and reflection and revision. These four phases involved 12 lecturers. The first phase was a 2-day workshop on TPACK. This workshop provided them with knowledge and skills on how to design English lesson plans. Two instructors led this introductory workshop. The first instructor was the researcher who introduced TPACK and technologies for ELT (academic blogs, oral/aural skill- building websites and bookmarks with delicious, reading/writing websites, and interactive PowerPoint), while the second instructor, the head of the language centre with expertise in teaching English to speakers of other languages (TESOL), presented on curriculum contents, strategies, and techniques in communicative language teaching (CLT). As part of the workshop the lecturers did micro-teaching in which they observed one another and then discussed their observations. The second phase consisted of lesson plan design. Participating English lecturers designed lesson plans for 1 week in groups of three, while the researcher facilitated and supported the process. The head of the language centre, the presenter in charge of curriculum content and pedagogy, facilitated the process and ensured that the designed lessons fit the curriculum. This phase was aimed at providing the English lecturers with the opportunity and experiences to design English lesson plans. The lesson plan format was decided together by the head of the language centre and the lecturers before the design. This produced an agreed upon format to be used at the language centre by all lecturers. The lesson activities were created in a flexible Australasian Journal of Educational Technology, 2015, 31(6). 703 format due to the lecturers’ other activities and conflicting teaching schedules at the centre. They met at least once every 2 days in the afternoon at the language centre office. The lessons they designed were based on the curriculum content from the textbooks used by students at the language centre. The lessons covered topics such as people and physical appearance, city life, and preserving the environment. The third phase was the implementation of the lesson plans. This implementation took place over 1 week. Each group taught the designed lessons in their classrooms. Classes were held two times a week. Each group had six class meetings to implement the lesson plans, and each class in general consisted of 30 to 35 students from a number of different fields of study. The last phase was reflection and revision. Upon the completion of this activity, the English lecturers reflected on the implemented lesson plans. At this point, they had an opportunity to exchange ideas in a roundtable discussion. This phase focused on which contents, technologies, and pedagogies helped students to reach the goals, how both students and lecturers achieved or failed to participate in the activities, what problems they faced, and how to best solve them. This reflective practice was conducted in meetings attended by all lecturers and the two facilitators. Afterwards, the lecturers revised the lesson plans based on the feedback and suggestions. A second phase of roundtable discussions was held to discuss and appraise the final designs. This whole phase lasted 1 week. Research methodology This study employed design-based research. It involved designing, implementing and evaluating a professional development programme intended to develop teachers’ knowledge and skills in technology integration in teaching English as a foreign language (TEFL). Among the produced are several design guidelines for a professional development programme that could become a basis for future professional development at the language centre. The intervention (the professional development programme) given may also position English lecturers as co-participants in the design and analysis to produce effective outcomes (Barab & Squire, 2004). Also, the intervention of this study was context-dependent. In other words, it could not be fully understood in separation from the language centre where the programme took place. In addition, it was strongly linked with the practice of English language teaching, thus having the potential to help develop more effective educational interventions and offer chances for professional learning for the lecturers during the research process (McKenney, Nieveen, & van den Akker, 2006). Therefore, the characteristics inherent in this study were deemed appropriate with design-based research. In addition, since design-based research has an iterative process that includes analysis, design and evaluation (McKenney et al., 2006), context analysis and literature review were conducted as part of the overall analysis. This allowed the researcher to understand the conditions necessary to design an intervention for technology integration. As a result, initial guidelines were produced for the organisation of the professional development arrangement. The second phase, design, consisted of a plan for the professional development arrangement. Based on the design guidelines, this study had two design cycles. The first cycle involved designing and developing an initial prototype of the professional development programme based on design guidelines generated from the context analysis and literature review. The second cycle involved systematic revision and improvement of the prototype. Formative evaluation was conducted to improve the programme, while summative evaluation was done to evaluate the impact of the programme. Both evaluations were reported in this study. Product quality In design-based research, quality in terms of validity, practicality and effectiveness should be established (Nieveen, 1999). In order to have high-quality professional development, the validity of this intervention was achieved through contextual analysis and review of contemporary literature about successful in-service teacher professional development programmes for technology integration. The practicality of the programme was also assessed by formative evaluations of the programme done with the English lecturers to find out their needs, suggestions, and difficulties throughout the intervention. Practicality was measured in relation to the lecturers’ experiences with the professional development programme as the first sub- research question addresses in cycle one. Finally, the effectiveness of the programme was evaluated through improvement of the instructors’ TPACK, attitude toward technology, and ability to design technology rich Australasian Journal of Educational Technology, 2015, 31(6). 704 lesson plans which then helped them in further real enactment in the language classrooms. The second sub- research question deals with this purpose in the cycle two. Participants All 12 lecturers who participated in the programme participated in this study. These participants were selected by the language centre based on their sufficient knowledge of computer application and internet skills, as well as their roles in the language centre. Instruments Interviews The interviews focused on questions about the TPACK professional development programme. The guiding questions of these interviews were piloted first to establish the reliability and understanding by the participants. Inter-rater reliability indicated an excellent agreement (.77) on data coding among the two assessors. However some overlapping questions and ambiguity were also found. The initial questions, which consisted of 11 guiding questions, were then reduced to 8. Furthermore, the interview was done after the first cycle and mainly conducted in the local language, Indonesian, to suit the participants’ preference. All interviews were recorded with the participants’ permission. Logbook A logbook was kept by the researcher to document the lecturers’ experiences with the programme in the first cycle. Field notes were taken throughout the entire programme. To make it easier for the researcher, the logbook was written in Indonesian and then translated into English after coding. Questionnaires Two types of questionnaires were administered to all English lecturers participating in this study immediately before and after the programme (intervention). A TPACK survey adapted from Schmidt et al. (2009) and a teacher attitude toward computers (TAC) survey adapted from Knezek (1998) were distributed to the English lecturers to identify their self-reported TPACK and attitudes before the programme in the first cycle, and after the programme in the second cycle. The TPACK survey was pilot tested with 10 English lecturers who did not participate in the professional development programme before being used with the research participants. Cronbach’s alpha for the survey with all its components (40 items) was .96, indicating a highly reliable questionnaire. Moreover, a TAC questionnaire with 22 items was also pilot-tested with the same participants. In general, the reliability of this questionnaire was .83 (very good). This indicates that this instrument was reliable. Rubric Since the TPACK and TAC questionnaires were not sufficient to determine the lecturers’ TPACK, lesson plan documents designed by the 12 lecturers before and after the second cycle of the professional development programme were analysed using an adapted technology integration assessment rubric from Harris, Grandgenett and Hofer (2010). This rubric was meant to assess the designed lessons before and after the intervention to investigate improvements in lesson plan design. The assessment involved two assessors, the researcher as the first assessor and the instructor for content and pedagogy as the second assessor who was trained by the researcher. The rubric has seven components and the maximum value of the TPACK lesson plan rubric was 21 points with 1 to 3 points for each criterion. Ten lesson plan samples were coded by two assessors. An inter-rater reliability was computed using the SPSS programme. In reference to Pedhazur and Schmelkin (1991), the result showed an excellent agreement between the two assessors, with a Cohen’s kappa of .87 (p = .000). Data analysis method The data collected for this study were analysed quantitatively and qualitatively. Data from interviews and the logbook were analysed using a data reduction technique: transcribing, generating categories through coding, and interpreting data. In the first process, the researcher transcribed the data that he collected especially from interviews. Both types of data were not translated into English to make them easy for the Australasian Journal of Educational Technology, 2015, 31(6). 705 researcher. After reading and re-reading, segmenting the transcriptions and the notes, data recorded in the logbook and transcriptions of the interviews were coded into categories and translated to English. This coding aimed at breaking the data into categories. As suggested by Miles and Huberman (1994), the researcher then generated the categories based on emerging themes. Afterwards, the coded data were entered into matrices to help view and interpret them. All matrices were given titles in line with the main categories. Data from TPACK and TAC questionnaires as well as lesson plan documents were analysed with a non-parametric method, the Wilcoxon signed ranks test, because the samples in this study could not be assumed to be normally distributed. This method was used to investigate differences in knowledge before and after the intervention. Trustworthiness The double role of the researcher in developing and conducting research activities may have generated a conflict of interests, a challenge in qualitative research as alerted to by van den Akker (1999). This was minimised by the researcher’s efforts to be as objective as possible. For example, the programme was comprehensively monitored by the language centre and all data were copied and made available for checking. In addition, lecturers can often be guided to respond positively when they know that they are under study: the so-called Hawthorne effect (Lodico, Spaulding, & Voegtle, 2006). In this case, such knowledge could have influenced their responses for the purpose of helping the researcher. To minimise the Hawthorne effect, the research questions of the study were not shared with the participants. Results The interview and logbook data revealed that all participants reported having positive experiences with the TPACK professional development programme. Seventy-five percent of respondents appreciated the programme, while few considered it as a tiring programme. The weaknesses of the programme were limited time for professional development and for technology exploration and lack of students’ active engagement. Moreover, all lecturers also reported that they better understood how to design lessons and changed the way they conceptualise their teaching, from seeing technology as a separate component to an integrated one. Eleven lecturers (91.6%) reported that the most beneficial phase was the design process. Suggestions were given about three major aspects: facilities and resources, follow-up, and support. Lecturers found difficulties in respect to limited facilities and resources and students’ ability. To cope with such barriers the lecturers facilitated lessons with their own resources, introduced the technology they used to support learning, and asked students to learn or explore the technology themselves. On the basis of the lecturers’ experiences, the professional development programme was improved with the refinement of the previous design guidelines based on the participants’ feedback. The improved guidelines became: 1. The use of an integrated framework provides teachers with a more accessible model for technology integration. The TPACK framework as suggested by Mishra and Koehler (2006, 2008) is deemed appropriate for describing the knowledge domains needed by lecturers for technology integration in EFL teaching. 2. A technology integration framework should place teachers in key roles in the process of meaningful learning. Since constructivism views learning as starting from such experiences and authentic learning, it can be adopted to provide meaningful learning for teachers. Koehler and Mishra’s (2005a) learning technology by design approach places English lecturers in the role of curriculum designers working in small collaborative teams to develop solutions to their real instructional problems. 3. Designing more effective professional development programmes requires that English lecturers be actively engaged in order that they become the main players in the programme and go through several authentic learning experiences in a collaborative environment: theoretical knowledge, course design, roundtable discussion, enactment of the designed course, and finally reflection and revision. 4. More practical and technical support should be given during the programme by technology, pedagogy, and technical computer experts. 5. The content of the programme should be based on the language centre curriculum currently used. Therefore, the textbooks should become the main source of content for the lecturers. Australasian Journal of Educational Technology, 2015, 31(6). 706 6. Feedback on lesson plans before the enactments should also be provided as learning guidance and critical inputs. Students could be invited to give suggestions. 7. The intervention should be extended to a 4-week professional development programme in order to strengthen lecturers’ learning and performance. The pre-survey conducted before the professional development programme showed that the lecturers (N = 12) reported a mean score of 68.7% in the mastery of all seven knowledge domains. The highest reported domain was PK, accounting for 82%, while the lowest percentage was found in TK, with 74%. Furthermore, lecturers also reported their TPCK which constituted 75% mastery. In conclusion, it seems that the lecturers need to develop their TK in particular and other domains in general in order to be able to connect content, pedagogy, and technology in an integrated manner. The data from TPACK questionnaires distributed after the programme demonstrated some increase on the lecturers’ knowledge domains. The lecturers (N = 12) reported a mean score of 87.7% in the mastery of all seven knowledge domains with an increase of 19% after the programme. The highest improvement was reported in the domain of TK, constituting a 16% increase, with a mean score of 31.91. TPK and TPCK had mean scores of 39.33 and 17.41 respectively, and had an increase of 12% each, thus putting these two domains as equal second highest in improvement. The lowest percentage of improvement was found in CK, with only 6%. In summary, it can be concluded that there is an overall improvement of the lecturers’ self- reported TPACK measures after participating in the professional development programme. Moreover, data from the pre and post TPACK surveys were compared (Table 1). They were calculated using SPSS. Based on the Wilcoxon signed ranks test (non-parametric test), the pre-scores and the post- scores are significantly different. Therefore, it can be concluded that the post-scores of TK, CK, PK, TCK, PCK, TPK, and TPCK are significantly higher than the pre-scores with p < 0.025 (one tailed). In other words, the results from the pre and post TPACK questionnaires show that the lecturers’ reported knowledge on TK, CK, PK, TCK, TPK, PCK, and TPCK increased after the TPACK professional development programme. The effect sizes were then measured. The data show that most effect sizes are above 0.50, except for TK which was only 0.48. Therefore, referring to Rosenthal (1984) benchmarks for interpreting effect sizes (r) (0.10 = small, 0.30 = medium, 0.50 = large), CK, PK, TCK, PCK, TPK, and TPACK had large effect sizes, while TK had a medium effect size (r = 0.48). The highest effect size was found in TPK, accounting for 0.59. Table 1 The summary statistics of the lecturers’ pre-treatment and post-treatment self-reported TPCK measures* Variable N Post-treatment Pre-treatment z Effect size p M SD M SD TK 12 25.58 3.553 23.25 4.901 -2.39 0.48 .008 CK 12 26.08 2.391 24.08 2.503 -2.53 0.51 .004 PK 12 31.91 2.151 28.91 2.391 -2.87 0.58 .001 PCK 12 17.75 1.288 15.75 0.866 -2.78 0.56 .002 TCK 12 18.08 1.880 14.83 3.069 -2.81 0.57 .001 TPK 12 39.33 3.200 34.16 5.441 -2.91 0.59 .001 TPCK 12 17.41 1.505 15.16 1.642 -2.79 0.56 .002 * Scores on TPACK measures were given on a 5-point Likert scale; higher scores reflect higher appreciation: strongly disagree = 1, disagree = 2, neither agree nor disagree = 3, agree = 4 and strongly agree = 5 Based on the lecturers’ self-reported TPACK measures, there was a difference between pre and post- treatments, indicating significant improvement after the programme for all components of TPACK. The Australasian Journal of Educational Technology, 2015, 31(6). 707 findings clearly confirm the prediction that TPACK would be enhanced by the presence of the TPACK professional development programme. The scores for the two conditions suggest that the effect sizes are relatively large for all domains, except in TK (medium). In addition to the lecturers’ self-reported TPACK measures, the lecturers’ lesson plans were also collected before the professional development programme. Their assessment was done by two evaluators who analysed seven components addressing TK, CK, PK, TCK, TPK, PCK, and TPCK. Each component was scored ranging from 1 (not at all), 2 (minimal) to 3 (strong). The criteria were based on the adapted technology integration assessment rubric from Harris, Grandgenett, and Hofer (2010). The overall findings from lesson plan assessment demonstrate that the TPACK professional development programme had a sound impact on lecturers’ lesson plan designs in all components of TPACK. However, only CK was not significant with z = -1.41 and p (0.250) > 0.025. A possible explanation for this is that it is probably due to the fact that the lecturers did not focus much on the curriculum content anymore, but rather were more concerned about aligning it with other knowledge domains, such as TK and PK. The data are detailed Table 2. Table 2 The summary statistics of the lecturers’ pre-treatment and post-treatment lesson plan design* Variable N Post-treatment Pre-treatment z Effect size P M SD M SD TK 12 3.00 0.000 1.37 0.678 -3.21 0.65 .000 CK 12 3.00 0.000 2.83 0.389 -1.41 ** .250 PK 12 3.00 0.000 2.37 0.226 -3.21 0.65 .000 PCK 12 2.75 0.452 1.58 0.514 -2.88 0.58 .001 TCK 12 2.75 0.452 1.00 0.000 -3.21 0.65 .000 TPK 12 2.75 0.452 1.00 0.000 -3.21 0.65 .000 TPCK 12 3.00 0.000 1.00 0.000 -3.46 0.70 .000 * Scores on lesson plan measures were given based on three criteria; higher scores reflect higher appreciation: not at all = 1, minimal = 2, and strong = 3 (p < 0.025) ** Not calculated because of insignificant p Both means of the pre and post results of lesson plan designs show different scores, according to the data. Based on the Wilcoxon signed ranks test, it can be concluded from the results shown in Table 2 that most of the knowledge domains improved as the result of the intervention. However, only CK is not significant with z = -1.41 and p (0.250) > 0.025. Therefore the effect size of this domain was not calculated. Effect sizes were also calculated based on the value of z, according to the Wilcoxon signed ranks test, and the results of the calculation demonstrate that the highest effect size was gained in TPCK, accounting for 0.70. The effect size of 0.65 was obtained from four knowledge domains: TK, PK, TCK, and TPK. All of these effect sizes are categorised as large, whilst the PCK effect size was 0.58 with z = -2.88 and p = 0.001. All these six domains indicate large effect sizes (Rosenthal, 1984). Finally, the overall findings showed that the TPACK professional development intervention for English language lecturers had a sound impact, as indicated by the overall result of the lecturers’ TPACK lesson plan designs and by their self-reported TPACK measures. Furthermore, data from the pre-surveys on the lecturers’ attitude toward computers (Table 3) testifies that the lecturers had good attitudes. This is indicated by their positive responses on the two sub-variables measured through TAC. It was evident, for example, that the lecturers’ enjoyment with computers had a mean score of 53.58 out of 60, while their anxiety indicated a mean score of 41.50 (83%). Therefore it appears that the lecturers were pleased to work with computers and were not worried about them. Australasian Journal of Educational Technology, 2015, 31(6). 708 Table 3 Summary statistics of the lecturers’ pre-treatment and post-treatment attitude toward computers* Variable N Post-treatment Pre-treatment z Effect size p M SD M SD Enjoyment 12 56.33 2.70 53.58 5.38 -2.32 0.47 .010 Anxiety 12 44.16 3.24 41.50 5.00 -1.73 ** .042 * Scores on TAC measures were also given on a 5-point Likert scale; higher scores reflect higher appreciation: strongly disagree = 1, disagree = 2, neither agree nor disagree = 3, agree = 4 and strongly agree = 5 (p < 0.025) ** Not calculated because of insignificant p The data obtained after the professional development programme indicate slight increases in both sub- variables. The mean score of enjoyment accounts for 56.33 (93%), thus indicating an increase of only 4% if compared with their initial score (M = 53.58). The lecturers’ anxiety, on the other hand, only decreased by 5%. It can be concluded that this perhaps happened because the lecturers already had a good attitude toward computers; they were also not anxious about using computers as shown previously. Conclusion and discussion This study aimed at putting relevant theories and guidelines into practice for a professional development programme for technology integration. It explored the potential of a professional development programme that adopts the TPACK framework and the learning technology by design approach for EFL lecturers. Conclusions were drawn from different data sources to investigate the effectiveness of the programme as measured from the lecturers’ TPACK development. In this case, data on self-reported TPCK was triangulated with the lecturers’ performance on lesson plan designs. The results of this study suggest that the TPACK professional development programme was successful, as indicated by the improvement of the lecturers’ TPCK and their positive experiences. This study confirmed the findings of previous studies (e.g., Allan et al., 2010; Blocher et al., 2011; Doering et al., 2009; Jimoyiannis et al., 2011; Tee & Lee, 2011). It can also be argued that the learning technology-by-design approach adopted in the programme could help lecturers develop TPCK in their English language teaching practices. As professional development is crucial for foreign language teaching reform, learning tasks given should be well suited to professional practices. A professional development programme with activities based on curriculum attainments and content is important to establish consistency of classroom practices. This provides lecturers with a relevant programme in line with the curriculum vision and content. In this regard, it is argued that teachers can see how to apply their professional learning activities to their teaching practices (Nicholas & Ng, 2012) and respect professional development programmes that connect their experiences and actual classroom practices (Garet, Porter, Desimone, Birman, & Yoon, 2001). In the case of the language centre, curriculum coherency did not only attract lecturers, but it also gained very strong support from the language centre since the programme was initiated by a researcher from the outside. This probably would have been different if it had been pioneered by the institution itself. Because the developer or researcher was not from the management at the centre, it was necessary for him to deeply understand the curriculum so as to establish relevance. By adopting the development stages from Joyce and Showers (1995), the model had sound learning activity sequencing. A similar finding was also reported by Bradshaw (2002), according to whom a positive effect was found by applying this model where teachers were involved in professional development activities that included theory, demonstration, practice, and follow-up. Teachers were more likely to transfer technology skills into teaching than those who participated in professional development activities that did not include all four dimensions. The introductory workshop on concepts such as TPACK gave the lecturers a model on how to integrate technology into English language teaching, and this became a foundation for their new knowledge as found in this study. Following this, demonstration led to a clearer application of the TPACK concept. In Merrill Australasian Journal of Educational Technology, 2015, 31(6). 709 (2002)’s first principles of instruction, the author advocated that learning is facilitated when learning activities also demonstrate the new knowledge to be learned rather than merely telling about what is to be learned. In this study, the lecturers were also asked to apply the model with which they practised designing English lessons in groups. In this regard, it was beneficial for the lecturers’ learning about technology integration to facilitate learning because the activity provided an opportunity for the lecturers to apply the new knowledge to new specific situations by designing specific concrete tasks (Merrill, 2002). Another important component was micro-teaching where the lecturers observed one another and then discussed their observations. All of these activities were organised in a collaborative learning environment. According to Joyce and Showers (1980), this learning environment is potential for teacher learning. Besides introducing the TPACK framework, the experiences with the enactment of the lessons had authenticated the lecturers’ design products. The implementation allowed the lecturers to test the design products and further reflect on their outcomes. Although students’ performance was not measured, they at least showed enthusiasm with their lecturers’ teaching. After the implementation, reflection on what had been done was conducted, and it formed the basis for redesigning the lessons in order to help students learn better. Overall, all these learning activities organised in a systematic way were critical for the lecturers’ TPCK development, for they fulfilled the English lecturers’ lack of knowledge and skills on technology integration. The main challenge in this workshop was to gather the lecturers because many of them had different activities at different times. This was resolved by grouping them into two workshop groups. Support from the facilitators and the institution was also needed to enhance the lecturers’ learning processes. Support from the institution and more knowledgeable people, as was suggested by the lecturers, helped create sustainable innovation and strengthen their conceptual and practical needs in the classrooms. Innovation initiated from external sources should convince and involve the key stakeholders in order that such innovation be institutionally accepted. During this study, guidance, feedback was given mostly in terms of pedagogical and technological support, as well as guidance during the lecturers’ discussions and design processes. Collaborative lesson plan design could promisingly assist English lecturers as they learn about technology integration, and it proves to be a good learning strategy for them to integrate technology into teaching. As this study reveals, collaboration created opportunities for lecturers to develop good lessons based on various perspectives, and they benefitted from each other’s knowledge and broad range of experiences. Handelzalts (2009) maintains that interactions during collaborative design allow teachers to investigate challenges to their instructional practice. Additionally, collaborative work decreases the lecturers’ load of accommodating both teaching duties and their professional development needs. As was advocated by Jonnassen, Mayes, and McAleese (1993), one of the most effective learning contexts is a collaborative environment. This is because teachers become team players and curriculum design peers (Simmie, 2007). The learning-technology-by-design approach proved to be effective in preparing English lecturers to integrate technology. This approach is a constructivist approach that views knowing as being situated in action and co-determined by individual environment interactions (cf. Koehler & Mishra, 2005a). Koehler and Mishra (2005a) argued that this approach is effective to develop deeper understanding of the complex interplay among technology, content, and pedagogy as well as the contexts where it is implemented. In this study, learning-technology-by-design put lecturers in the centre of the activities as the designers of the lesson plans. Finally, the design guidelines produced in this study constitute a salient foundational framework to design professional development projects for technology integration. However, the guidelines in this study should not be regarded as a recipe, but rather as tools to help others select and apply the most appropriate knowledge for specific design and development tasks in their own settings (McKenney et al., 2006). As the findings suggest, this type of intervention can have a sound impact on lecturers’ TPCK development. Acknowledgement The author would like to thank Prof Joke Voogt and Dr Petra Fisser for their critical insights, and also gratefully acknowledge the helpful comments from anonymous reviewers. Finally, he also thanks Fabio Coelho for his assistance in proofreading this paper. Australasian Journal of Educational Technology, 2015, 31(6). 710 References Allan, W. C., Erickson, J. L., Brookhouse, P., & Johnson, J. L. (2010). EcoScienceWorks: Teacher professional development through a collaborative curriculum project-an example of TPACK in Maine. TechTrends, 54(6), 36-43. doi:10.1007/s11528-010-0452-x Angeli, C., & Valanides, N. (2009). Epistemological and methodological issues for the conceptualization, development, and assessment of ICT-TPCK: Advances in technological pedagogical content knowledge (TPCK). Computers and Education, 52(1), 154-168. doi:10.1016/j.compedu.2008.07.006 Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1-14. doi:10.1207/s15327809jls1301_1 Blocher, J. M., Armfield, S. W., Sujo–Montes, L., Tucker, G., & Willis, E. (2011). Contextually based professional development. Computers in the Schools, 28(2), 158-169. doi:10.1080/07380569.2011.577398 Bradshaw, L.K. (2002). Technology for teaching and learning: Strategies for staff development and follow-up support. Journal of Technology and Teacher Education, 10(1), 131-150. Norfolk, VA: Society for Information Technology & Teacher Education. Cox, S., & Graham, C. R. (2009). Diagramming TPACK in practice: Using an elaborated model of the TPACK framework to analyze and depict teacher knowledge. TechTrends, 53(5), 60-69. doi:10.1007/s11528-009-0327-1 Doering, A., Veletsianos, G., Scharber, C., & Miller, C. (2009). Using the technological, pedagogical, and content knowledge framework to design online learning environments and professional development. Journal of Educational Computing Research, 41(3), 319-346. doi:10.2190/EC.41.3.d Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915-946. doi:10.3102/00028312038004915 Guzey, S. S., & Roehrig, G. H. (2009). Teaching science with technology: Case studies of science teachers’ development of technology, pedagogy, and content knowledge. Contemporary Issues in Technology and Teacher Education, 9(1), 25-45. Retrieved from http://www.citejournal.org/articles/v9i1science1.pdf. Handelzalts, A. (2009). Collaborative curriculum development in teacher design teams (Doctoral dissertation). University of Twente, Enschede. Harris, J., Grandgenett, N., & Hofer, M. (2010). Testing a TPACK-based technology integration assessment instrument. In C. D. Maddux, D. Gibson, & B. Dodge (Eds.), Research highlights in technology and teacher education 2010 (pp. 323-331). Chesapeake, VA: Society for Information Technology and Teacher Education (SITE). Harris, J. B., & Hofer, M. J. (2009). Instructional planning activity types as vehicles for curriculum-based TPACK development. In C. D. Maddux (Ed.), Research highlights in technology and teacher education (pp. 99-108). Chesapeake, VA: Society for Information Technology in Teacher Education (SITE). Harris, J. B., & Hofer, M. J. (2011). Technological Pedagogical Content Knowledge (TPACK) in action: A descriptive study of secondary teachers’ curriculum-based, technology-related instructional planning. Journal of Research on Technology in Education, 43(3), 211-229. doi:10.1080/15391523.2011.10782570 Haydn, T., & Barton, R. (2008). ‘First do no harm’: Factors influencing teachers’ ability and willingness to use ICT in their subject teaching. Computers & Education, 51(1), 439-447. doi:10.1016/j.compedu.2007.06.001 Hew, K. F., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Education Technology Research and Development, 55, 223–252. doi:10.1007/s11423-006-9022-5 Jimoyiannis, A. (2010). Designing and implementing an integrated technological pedagogical science knowledge framework for science teachers professional development. Computers and Education, 55(3), 1259-1269. doi:10.1016/j.compedu.2010.05.022 Jimoyiannis, A., Tsiotakis, P., & Roussinos, D. (2011). Pedagogical and instructional design issues towards the integration of Web 2.0 tools in instruction - Implications of teachers' training pilot courses in Greece. Paper presented at the 7th International Conference on Next Generation Web Services Practices. doi:10.1109/NWeSP.2011.6088235 Australasian Journal of Educational Technology, 2015, 31(6). 711 Jonnassen, D. H., Mayes, T., & McAleese, R. (1993). A manifesto for a constructivist approach to uses of technology in higher education. In T. M. Duffy, J. Lowyck, & D. H. Jonassen (Eds.), Designing environments for constructivist learning (pp. 231-247). Berlin: Springer-Verlag. Joyce, B., & Showers, B. (1980). Improving in-service training: The messages of research. Educational Leadership, 37(5), 379-385. Joyce, B., & Showers, B. (1995). Student achievement through staff development: Fundamentals of school renewal (2nd ed.). New York, NY: Longman. Knezek, G. (1998). Parallel forms for measuring teachers’ attitudes toward. In S. McNeil, J. Price, S. Boger-Mehall, B. Robin & J. Willis (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 1998 (pp. 832-836). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE). Retrieved from http://www.editlib.org/p/47538. Koehler, M. J., & Mishra, P. (2005a). Teachers learning technology by design. Journal of Computing in Teacher Education, 21(3), 94-102. doi:10.1080/10402454.2005.10784518 Koehler, M. J., & Mishra, P. (2005b). What happens when teachers design educational technology? The development of technological pedagogical content knowledge. Journal of Educational Computing Research, 32(2), 131-152. doi:10.2190/0EW7-01WB-BKHL-QDYV Koehler, M. J., & Mishra, P. (2008). Introducing TPACK. In AACTE committee on innovation & technology (Eds.), Handbook of technological pedagogical content knowledge for educators (pp. 3– 29). New York, NY: Routledge. Koehler, M. J., Mishra, P., & Yahya, K. (2007). Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy and technology. Computers & Education, 49(3), 740- 762. doi:10.1016/j.compedu.2005.11.012 Lodico, M. G., Spaulding, D. T., & Voegtle, K. H. (2006). Method in educational research: From theory to practice. San Fransisco, CA: Jossey-Bass. McKenney, S., Nieveen, N., & van den Akker, J. (2006). Design research from a curriculum perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research. London: Routledge. Merrill, M. D. (2002). First principles of instruction. ETR&D, 50(3), 43-59. doi:10.1007/BF02505024 Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis. Thousands Oaks, CA: SAGE Publications. Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054. Retrieved from http://punya.educ.msu.edu/publications/journal_articles/mishra-koehler-tcr2006.pdf. Nicholas, H., & Ng, W. (2012). Factors influencing the uptake of a mechatronics curriculum initiative in five Australian secondary schools. International Journal of Technology and Design Education, 22(1), 65-90. doi:10.1007/s10798-011-9156-6 Niess, M. L., Zee, E. H., & Gillow-Wiles, H. (2010). Knowledge growth in teaching mathematics/science with spreadsheets: Moving PCK to TPACK through online professional development. Journal of Digital Learning in Teacher Education, 27(2), 42-52. doi:10.1080/21532974.2010.10784657 Nieveen, N. (1999). Prototyping to reach product quality. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 125-135). Dordrecht: Kluwer Academic Publishers. Oster-Levinz, A., & Klieger, A. (2010). Online tasks as a tool to promote teachers' expertise within the technological pedagogical content knowledge (TPACK). Procedia Social and Behavioral Sciences, 2, 354-358. doi:10.1016/j.sbspro.2010.03.024 Pedhazur, E. J., & Schmelkin, P. (1991). Measurement, design and analysis. Hillsdale, NJ: Lawrence Erlbaum. Pineida, F. O. (2011). Competencies for the 21st century: integrating ICT to life, school and economical development. Procedia - Social and Behavioral Sciences, 28(0), 54-57. doi:10.1016/j.sbspro.2011.11.011 Rosenthal, R. (1984). Applied social research methods series, Vol. 6: Meta-analytic procedures for social research. Newbury Park, CA: Sage Publications. Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technological pedagogical content knowledge (TPACK): The development and validation of an assessment instrument for preservice teachers. Journal of Research on Technology in Education, 42(2), 123-149. doi:10.1080/15391523.2009.10782544 Australasian Journal of Educational Technology, 2015, 31(6). 712 Simmie, G. M. (2007). Teacher design teams (TDTs)—Building capacity for innovation, learning and curriculum implementation in the continuing professional development of in career teachers. Irish Educational Studies, 26(2), 163-176. doi:10.1080/03323310701295914 Tee, M. Y., & Lee, S. S. (2011). From socialisation to internalisation: Cultivating technological pedagogical content knowledge through problem-based learning. Australasian Journal of Educational Technology, 27(1), 89-104. Retrieved from http://ajet.org.au/index.php/AJET/article/view/984/257 Thompson, A. D., & Mishra, P. (2007). Breaking news: TPCK becomes TPACK! Journal of Computing in Teacher Education, 24(2), 38-64. doi:10.1080/10402454.2007.10784583 Tondeur, J., van Keer, H., van Braak, J., & Valcke, M. (2008). ICT integration in the classroom: Challenging the potential of a school policy. Computers and Education, 51(1), 212-223. doi:10.1016/j.compedu.2007.05.003 Trautmann, N. M., & MaKinster, J. G. (2010). Flexibly adaptive professional development in support of teaching science with geospatial technology. Journal of Science Teacher Education, 21(3), 351-370. doi:10.1007/s10972-009-9181-4 Universitas Islam Negeri Sultan Syarif Kasim Riau. (2007). Rencana induk pengembangan teknologi informasi (IT strategic planning). Pekanbaru: Universitas Islam Negeri Sultan Syarif Kasim Riau. van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, R. Maribe Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1-14). Dordrech: Kluwer Academic Publishers. Warschauer, M., & Kern, R. (2000). Network-based language teaching: Concepts and practice. New York, NY: Cambridge University Press. Warschauer, M., & Meskill, C. (2000). Technology and second language teaching and learning. In J. W. Rosenthal (Ed.), Handbook of undergraduate second language education (pp. 303-318). Mahwah, NJ: Lawrence Erlbaum Associates. Young, S. S. C. (2003). Integrating ICT into second language education in a vocational high school. Journal of Computer Assisted Learning, 19(4), 447-461. doi:10.1046/j.0266-4909.2003.00049.x Corresponding author: Muhammad Fauzan Ansyari, m.f.ansyari@uin-suska.ac.id Australasian Journal of Educational Technology © 2015. Please cite as: Ansyari, M. F. (2015). Designing and evaluating a professional development programme for basic technology integration in English as a foreign language (EFL) classrooms. Australasian Journal of Educational Technology, 31(6), 699-712. mailto:m.f.ansyari@uin-suska.ac.id Introduction Literature review Framework for teacher knowledge base in technology integration Figure 1. TPACK framework and its knowledge components (www.tpack.org) Learning technology by design approach Design of the intervention Initial design guideline Development of the professional development programme Research methodology Product quality Participants Instruments Interviews Logbook Questionnaires Rubric Data analysis method Trustworthiness Results Conclusion and discussion Acknowledgement References