Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) REiD (Research and Evaluation in Education), 7(1), 2021, 46-56 Available online at: http://journal.uny.ac.id/index.php/reid Evaluation of learning process: Knowledge of ICT integration among pre- service English language teachers Dyah Setyowati Ciptaningrum; Nur Hidayanto Pancoro Setyo Putro; Nila Kurnia Sari*; Nurqadriyanti Hasanuddin Universitas Negeri Yogyakarta Jl. Colombo No. 1, Karangmalang, Depok, Sleman, Yogyakarta 55281, Indonesia *Corresponding Author. E-mail: nilakurniasari@uny.ac.id INTRODUCTION ICT has the potentials to assist learners in the process of acquiring English language pro- ficiency. For example, it can bring them the immersion-like language learning experiences. ICT also has the tools to provide foreign language learners with authentic language learning materials, authentic cultural context, and meaningful human interactions which can enhance motivation for language practice and the development of students‟ language skills (Golonka et al., 2014; Kern et al., 2017). In addition to this, ICT has made the customization of learning possible (Ahmed et al., 2020). Therefore, ICT also offers customized language learning experience. Another advantage that can be offered by using ICT in English Language Teaching (ELT) is that students are facilitated to acquire the 21st century learning skills. These skills become an ARTICLE INFO ABSTRACT Article History Submitted: 06 March 2020 Revised: 28 June 2021 Accepted: 29 June 2021 Keywords technological and pedagogical content knowledge; TPACK; pre- service teacher Scan Me: This research aims at investigating the knowledge of ICT integration among the stu- dents of the English Language Education Study Program at a particular university in Yogyakarta. By employing the quantitative method, this ex-post facto research in- volved 70 English Department students who had taken microteaching as respondents in a survey to gather the data using a questionnaire. The questionnaire was taken from the model developed by one of the researchers based on the TPACK framework. There are five domains in the questionnaire. Four domains measure TPACK percep- tions: Technological Knowledge (TK), Technological Content Knowledge (TCK), Technological Pedagogical Knowledge (TPK), and Technological Pedagogical Content Knowledge (TPACK). One domain measures the pre-service teachers‟ perceptions of their ICT-related learning experiences. Demographic questions are included to identify the characteristics of the respondents in order to understand gender differences or re- lationships between teachers who have access to technologies at home and those who do not. The closed-ended questions applied a five-point Likert-type scale ranging from 1=strongly disagree to 5=strongly agree. Correspondingly, it contains one open-ended question in the TPACK domain. Descriptive analysis and MANOVA were conducted for the quantitative data analysis. The result of this study showed that the pre-service students possess a high self-confidence in the application of TK (Technological Knowledge), TPK (Technological Pedagogical Knowledge), and TCK (Technological Content Knowledge). However, their TPACK appears to need further development. The results from MANOVA showed that the pre-service teachers‟ related learning ex- periences significantly differed by the students‟ gender, with male students reporting more experiences than female students. This is an open access article under the CC-BY-SA license. How to cite: Ciptaningrum, D., Putro, N., Sari, N., & Hasanuddin, N. (2021). Evaluation of learning process: Knowledge of ICT integration among pre-service English language teachers. REID (Research and Evaluation in Education), 7(1), 46- 56. doi:https://doi.org/10.21831/reid.v7i1.30521 https://creativecommons.org/licenses/by-sa/4.0/ https://doi.org/10.21831/reid.v7i1.30521 https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 47 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) important aspect of employability since what companies need from its workers today are their abilities to be flexible and adaptable; to have initiative and self-direction; to show social and cross-cultural skills; to be productive and accountable; and to show leadership capacity and re- sponsibility (Keengwe, 2013; Koh et al., 2015). ICT needs to be used as a tool to foster meaning- ful learning where learners are engaged in a critical thinking process (Bryan & Wang, 2013). Despite all the advantages, it is still quite a challenge for educators to fully optimize the application of technology in practice. In fact, the use of ICT in classroom has been criticized as being too focused on transferring information as opposed to helping learners in constructing knowledge (Turgut, 2017). Similarly, research by Hutchison and Reinking (2011) and Tour (2015) reported that in practice, language teachers have not been fully utilizing technology to reinforce the delivery of instruction and the application of curriculum, and that the use of technology in teaching language has been superficial. A survey study on K-12 foreign language teachers in the U.S. by Rosetta Stone and Project Tomorrow (2013) revealed that the majority of the teachers had been incorporating technology in their classroom. However, up to 88% of them had been using technology only for searching resources in the internet. This is far from realizing the full potential of ICT in foreign language class because as Ding et al. (2019) pointed out, ICT may present a 'unique opportunity' for language learners to learn cultures and 'interpersonal commu- nication'. Thus, the use of ICT in language class should be more than just for accessing and pre- senting information. The potentials of ICT in ELT can only be realized if foreign language teachers/instructors can use ICT well (Alkamel & Chouthaiwale, 2018; Mei et al., 2018; Taopan et al., 2020). It means that merely knowing how to operate the ICT hardware and software is inadequate for EFL teachers. This knowledge needs to be connected with curriculum and good pedagogy (Raygan & Moradkhani, 2020; Tondeur et al., 2013). As Turgut (2017, p. 4) pointed out, it is necessary for teachers and preservice teachers to be well-trained in the use of technology in regard to the fol- lowing domains: '...knowledge, skills, and attitudes'. Teachers‟ ability to design learning environ- ments with authentic learning tasks that utilize ICT as a tool where students are engaged in col- laborative activities that address individual student‟s learning style, needs, and interest is essential to successful ICT integration (Majumdar, 2015). The Indonesian Ministry of Education has mentioned that Indonesian teachers, including English language teachers, need to integrate ICT in the learning and teaching process (Regulation of the Minister of National Education No. 16 of 2007; Regulation of the Minister of National Education No. 41 of 2007; Regulation of the Minister of National Education No. 78 of 2009). If the Ministry of National Education requires Indonesian teachers to integrate ICT in the learning and teaching process, then the role of EFL pre-service teacher education becomes crucial as it serves as the initial and primary source of EFL teachers‟ knowledge. EFL teacher education pro- grams need to extend EFL teachers‟ knowledge base to incorporate a set of knowledge on how to link technology with the nature English language as the subject matter and the appropriate strategies in ELT that are suitable with their classroom and school‟s context (Ahmadi, 2018; Gönen, 2019). What EFL teachers learned during their pre-service study would influence the way they teach as in-service teachers. Previous researches revealed that there is a correlation between ICT knowledge and how preservice teachers apply technology in their classroom. As reported by Kartchava and Chung (2015) and Turgut (2017), preservice teacher who were highly trained in the use of ICT to facilitate classroom activities are more inclined to use technology to reinforce their own class. This statement highlights the importance of ensuring that preservice teachers receive the best ICT education prior to entering workforce. On this note, teachers who belief that learning process should be learner-centered were also found to be more receptive towards incorporating technology into their class (Deng et al., 2014). Have Indonesian EFL pre-service teachers, i.e. the Indonesian EFL pre-service teachers at this particular university in Yogyakarta, been prepared to teach using ICT? There have been a number of studies that measure the https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 48 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) development of teachers‟ TPACK (Chai et al., 2010; Ersanli, 2016; Koehler et al., 2007; Kurt et al., 2014; Kwangsawad, 2016; Öz, 2015). However, little is known on the TPACK level of the Indonesian EFL pre-service teachers. This study aims at examining the perception of Indonesian EFL pre-service teachers in this particular university in Yogyakarta on their knowledge in apply- ing ICT-mediated instructions through the lens of Mishra and Koehler‟s model of TPACK. Their level of TPACK is measured and observed. The result of the study will inform the development of a particular instructional design to assist EFL pre-service teachers in designing an ICT-me- diated instruction. This study will also expand knowledge on pre-service teachers‟ TPACK in de- veloping countries through a particular case. While the Introduction section covers the overview to this research, this section reviews previous research on the teachers‟ TPACK. This literature review aims to establish the relationships of this research to the existing theory and other related research. It also informs the rest of the design in this research. Krashen's (1985) Input Hypothesis has been the foundation of recognizing the contribu- tion of ICT in language learning. In this theory, interaction provides opportunities for second language learners to understand messages (comprehensible input) to develop learners‟ second language (Brown, 2000). Swain and Lapkin (1995) criticised this Input Hypothesis by arguing that output was central in developing learners‟ competence in the target language. The Output Hypo- thesis views that language learners realize the gaps in their linguistic knowledge based on the external and internal feedback of the language that they produced. Pica (1994) stated that in the process of interaction, language learners have various opportunities for receiving comprehensible input and to test their linguistic knowledge based on the interlocutors‟ feedback. Thus, this kind of interaction which provides comprehensible input and output for language learners is present in the total-immersion situation (Goodwin, 1991). In this situation, language learners received the language exposure to develop their linguistic knowledge and they are forced to use the language actively if they want to survive in this environment because their surrounding context is written in the target language and the people speak the target language. Another concept that underpins the application of ICT in language learning is the collabo- rative learning principle which has its root in Vygotsky‟s sociocultural theory of mind. Knowl- edge acquisition, in this sociocultural learning theory, is not an individual pursuit but its devel- opment is mediated by symbolic means (Lantolf, 1994). Language and technology are two exam- ples of the symbolic means as they allow humans to organize their mental and physical activities. Learning occurs as a result of interaction between the learner‟s biological and sociocultural fac- tors in the learner‟s environment. Social interaction, then, has a significant role in creating an en- vironment to learn language. Collaborative learning provides a means toward students‟ cognitive development since assistance from other people who are more skilful and experienced is available (Warschauer, 1997). Since the 1980‟s, a rich body of research has been conducted on the role of ICT in affect- ing language learning. The early uses of CALL were mainly limited in the form of drill and practice activities and with the rapid growth of technologies, communicative and integrated uses of CALL were emerging (Liu et al., 2002). As the advent of technologies are giving ways to the widespread use of Web 2.0 tools, recent studies (e.g., Amir et al., 2011; Kılıçkaya & Krajka, 2012; Mirzaei et al., 2016; Wang et al., 2014) have also confirmed ICT to be effective in influencing language learning. TPACK, which stands for Technological Pedagogical Content Knowledge, is a framework developed by Mishra and Koehler (2006). This framework is an extension of Shulman's (1986) concepts of Pedagogical Content Knowledge (PCK). As teaching practice developed, Shulman's (1986, 1987) notion of pedagogical content knowledge as teachers‟ knowledge base needs to be expanded to include knowledge of ICT use in education to address the constraints in the ICT in- tegration mentioned earlier. Thus, Mishra and Koehler (2006) included one more dimension into the framework: Technological Knowledge and discovered that more types of knowledge can exist between the three domains of knowledge, as presented in Figure 1. https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 49 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) Figure 1. TPACK Framework by Mishra and Koehler (2006) Technology Knowledge (TK) refers to the skills to use the technology. Teachers need to show the ability to use the standard technology like the black/white board, textbooks, visual aids, or the new technology like the Internet and digital video. Including in this knowledge are teach- ers‟ skills to operate computer system and hardware, and use software tools like word processors, powerpoint, spreadsheet, web browsers, e-mail, and instant messaging. Digital technology is continuously changing. It is imperative for teachers to have the ability to keep up and adapt with the changes in technology. In addition, teachers should also need to decide whether the technol- ogy supports or hinders the attainment of the purpose of the lesson (Koehler et al., 2014). Technological Content Knowledge (TCK) includes the ability to select the appropriate technology tool to deliver the subject matter since technology can support or impede the learning of the subject matter. The nature of the ideas in the subject matter drives the selection process. TCK for foreign language teachers can be defined as “the body of knowledge that teachers have about their target language and its culture and how technology is used to represent this knowl- edge” (van Olphen, 2008, p. 113). Technological Pedagogical Knowledge (TPK) is the interaction between technology and pedagogy. Teachers have a repertoire of teaching strategies and they should be able to skilfully select the one that best represents the idea in the subject matter and suits the students‟ context or characteristics such as age, fluency/mastery level of the topic, learning style, or background knowledge. With technology, the complexity increases. Teachers need to understand how tech- nology can change the teaching and learning. There are different technology tools that can be used for a task. The selection of the appropriate tool is “based on its fitness, strategies for using the tool‟s affordances, and knowledge of pedagogical strategies and the ability to apply those stra- tegies for use of technologies. This includes knowledge of tools for maintaining class records, attendance, and grading, and knowledge of generic technology-based ideas such as WebQuests, discussion boards, and chat rooms” (Mishra & Koehler, 2006, p. 1028). Technological Pedagogical Content Knowledge (TPACK) is the heart of effective teaching using technology. It requires “an understanding of how to represent concepts with technologies, pedagogical techniques that use technologies in constructive ways to teach content; knowledge of what makes concepts difficult or easy to learn and how technology can help students learn; knowledge of students‟ prior knowledge and theories of epistemology; and knowledge of how technologies can be used to build on existing knowledge and to develop new epistemologies or strengthen old ones” (Mishra & Koehler, 2008, p. 10). https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 50 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) Using Mishra and Koehler‟s concept of TPACK, van Olphen (2008, p. 117) believes that meaningful technology integration in language teaching entails the following condition: (1) an un- derstanding of how linguistic and cultural concepts can be represented using technology, (2) edu- cational approaches to language teaching that draw from socio-constructivist philosophies to de- velop students‟ language and cultural competence, (3) an awareness of what facilitates or hinders the acquisition of language and the development of language competence and how technology, specifically CALL or CMC, can revamp common problems that students ordinarily face, (4) an awareness of students‟ previous knowledge, and particularly a knowledge of second language ac- quisition and cognitive development theories, (5) an understanding of how current and emerging technologies can be used to advance present knowledge and to develop new epistemologies and sustain previous ones. The three domains in Shulman‟s framework are not exclusive. Tseng (2016) explained that the relationship between the three components (technological, pedagogical, and content) in the theoretical framework of TPACK is 'dynamic' and 'transactional' and that it is especially impor- tant for teachers to be aware of these natures. Lack of awareness in the nature of TPACK may result in treating the domains as separated knowledge when in fact, all TPACK domains have overlapping sections and are interchangeable. METHOD The research took place at the English Language and Education Department at a univer- sity in Yogyakarta. The initial sample included 70 pre-service teachers who sit in Micro-teaching classes during March 2017 to August 2017 (fifth semester students). A TPACK survey was ad- ministered at the end of the semester to obtain a description on their level of TPACK before they left for a teaching practicum at selected schools. A set of questionnaires has been developped, and it had undergone a validity and reliability study (Ciptaningrum, 2017). This set of questionnare consists of three questions under the Tech- nological Knowledge (TK) section, ten questions for Technological Content Knowledge (TCK) section, and six questions under the Technological Pedagogical Knowledge (TPK) section. All of the questions are closed-ended. For Technological Pedagogical Content Knowledge (TPACK), there are six closed-ended questions and one open-ended question. All questions were uploaded using Google Form platform. It was pilot tested three times and revised before the final version was uploaded. At the beginning of July 2017, an invitation was sent to the whole cohort to participate in the survey through the WhatsApp Group of each class. From the total of 110 students who en- rolled in this topic/subject, 70 students responded, constituting a response rate of 63.6%. The data were analyzed based on descriptive statistics by applying Central Tendency Measures to find the mean. Then, the qualitative data conversion for likert scale questionnaire was used to classify the mean. Thus, the TPACK level of the pre-service teachers will be revealed, whether it is „very poor‟, „poor‟, „fair‟, „good‟, or „very good‟. Then, prior to the MANOVA test, the Box‟s Test of Equality of Covariance Matrices was conducted to check the assumption of homogeneity of co- variance across the groups using p < .001 as a criterion. FINDINGS AND DISCUSSION Findings The mean score of each question under the Technological knowledge (TK) showed that pre-service teachers are generally confident with their TK: Q1 (M = 4.12, sd = 0.74), Q2 (M = 3.74, sd = 1.03), and Q3 (M = 3.97, sd = 0.99). Different types of technology, both hardware and software, are familiar to them since opportunities to „play around‟ with different technology are available. https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 51 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) Similarly, they also have confidence in using technology to help them build the content knowledge required as future English language teachers (TCK). The mean score for each ques- tion under this section are: Q4 (M = 3.98, sd = 0.94), Q5 (M = 3.39, sd = 0.91), Q6 (M = 4.02, sd = 0.85), Q7 (M = 4.47, sd = 0.65), Q8 (M = 4.60, sd = 0.68), Q9 (M = 4.28, sd = 0.76), Q10 (M = 4.21, sd = 0.83), Q11 (M = 3.82, sd = 0.91), Q12 (M = 4.05, sd = 0.86), Q13 (M = 4.07, sd = 0.82). As for TPK, the data show that the pre-service teachers are also confident with their knowledge on using ICT for pedagogical reasons. The mean score for the TPK questions are: Q14 (M = 4.11, sd = 0.82), Q15 (M = 3.77, sd = 0.93), Q16 (M = 3.72, sd = 0.89), Q17 (M = 3.78, sd = 0.84), Q18 (M = 3.98, sd = 0.87), Q19 (M = 3.95, sd = 0.85). The TPACK questions yields the following result: Q21 (M = 3.97, sd = 0.68), Q22 (M = 3.71, sd = 0.81), Q23 (M = 3.82, sd = 0.72), Q24 (M = 3.50, sd = 0.79), Q25 (M = 3.60, sd = 0.84), Q26 (M = 3.47, sd = 0.84). This indicates that the pre-service teachers are less confident about their knowledge on efective ICT integration. In short, the pre-service teachers‟ TK, TCK and TPK in integrating ICT into English lan- guage teaching and learning are classified as „good‟ with the mean scores after qualitative data conversion of 3.94, 4.08, and 3.88 respectively. However, their TPACK falls under the category „fair‟ with a mean score of 3.67. Next, the Box‟s Test of Equality of Covariance Matrices was run to explore the assumption of homogeneity of covariance across the groups using p < .001 as a criterion. The results showed that the Box‟s M (19.55) was not significant, p (.343) indicating that there are no significant dif- ferences between the covariance matrices. Subsequently, MANOVA was conducted with gender as the independent variable and the five dimensions of knowledge of ICT integration as the de- pendent variables. The results show significant associations between gender and the five dimen- sions of knowledge of ICT integration. In other words, there was a statistically significant differ- ence in the five dimensions of knowledge of ICT integration based on the students gender, F (5, 64) = 3.58, p < .01; Wilk's Λ = 0.781. The multivariate effect size, as presented in Table 1, was estimated at .219, which implies that 21.9 % of the variance in the canonically derived dependent variable, i.e. the five dimensions of knowledge of ICT integration were accounted for by gender. Table 1. Results of Multivariate Tests Effect Value F Hypothesis df Error df Sig. Partial Eta Squared Gender Pillai‟s Trace .219 3.582b 5.000 64.000 .006 .219 Wilks‟ Lambda .781 3.582b 5.000 64.000 .006 .219 Hotelling‟s Trace .280 3.582b 5.000 64.000 .006 .219 Roy‟s Largest Root .280 3.582b 5.000 64.000 .006 .219 a. Design: Intercept + gender b. Exact statistic Table 2. Results of Tests of Between-Subjects Effects Source Dependent Variable Type III Sum of Squares df Mean Square F Sig. Gender TK .089 1 .089 .042 .839 TCK 34.300 1 34.300 1.074 .304 TPK 1.289 1 1.289 .095 .758 TPCK 4.629 1 4.629 .360 .551 TRLE 68.014 1 68.014 13.768 .000 A closer look at the results of the test between subjects effects as presented in Table 2 showed that gender has a statistically significant effect on pre-service English language teachers‟ related to learning experiences (F (1, 68) = 13.77; p < .01) with male pre-service teachers report- ed more experiences (M= 17.00, SD= 1.66) compared to their female counterparts (M= 14.54, SD= 2.33). https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 52 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) Discussion Studies on teachers‟ learning on the use of ICT (e.g. Alelaimat et al., 2020; Ciptaningrum, 2018; Yet & Noordin, 2017) conclude that ICT Professional Development (PD) for teachers comprise of standardize, one-size-fit all programs that show teachers how to use the technology hardware and software. Pre-service teachers at the English Language and Education Depart-ment at this university in Yogyakarta, have received training on how to use an online learning manage- ment system for learning purposes. In addition, most of them tend to use ICT on daily bases. They also have taken topics on content and pedagogy as parts of curriculum at this institution. This explains their confidence in their TK, TCK, and TPK. TPK and most importantly TPACK are important because in planning their lessons, teach- ers should not only include the selection of technology tools that can best represent difficult con- cepts, but they also need to understand how these technology tools can afford the pedagogy tech- niques that are student-centred, authentic, facilitate meaningful learning (Mishra & Koehler, 2008; van Olphen, 2008), and develop students‟ English language ability, not only in reading and listen- ing, but also in speaking and writing skills. Besides, van Olphen (2008) had outlined the components of TPACK in the context of for- eign language education (refer to section 2.2). Thus, developing students‟ communicative skills in the target language will benefit from using the technology in the way that this technology does not only represent linguistic concepts, but they also need to facilitate teachers to introduce cul- ture to their students (Blake, 2013), and support educational approaches to pedagogy that draw from socio-constructivist philosophies (Hidayati, 2016; Kern et al., 2017). This TPACK develop- ment needs to be the framework which underlines the English language pre-service teachers‟ ICT PD. Related to the gender differences, there is no statistically significant difference between the males and females regarding to their TPACK. However, in terms of Technology-related learning experiences, the male students were reported to have more learning experiences than the female students. They learn the use of technology from various sources: friends, enrolling in a technol- ogy course inside and outside university, reading books or magazines on technology and attend- ing workshops, seminars or conference on technology. Most of them also provide additional comments that they learn the use of new apps by themselves, through YouTube tutorials and they are not afraid to make mistakes. Regarding the gender differences in the technological re- lated learning experience in this research, it tends to support some empirical studies that found that women have a low significance for technological participation than men due to social cul- tural attitudes about the role of women in society (Antonio & Tuffley, 2014). However, further investigation need to be conducted to confirm this. CONCLUSION Pre-service teachers at this English Language and Education Department, in a university in Yogyakarta, have higher confidence in using the technoly hardware and software, as well as using ICT to deliver the subject matter. They also understand how technology can change the teaching and learning. However, it seems that they need more assistance to learn how ICT can be inte- grated effectively to enhance students learning and improve their English language skills. Further studies might need to confirm the result of this study‟s finding qualitatively. Furthermore, studies on developing an instructional model of ICT integration in English language classrooms need to be conducted before analyzing the implementation result of the model. REFERENCES Ahmadi, M. R. (2018). The use of technology in English language learning: A literature review. International Journal of Research in English Education, 3(2), 115–125. https://doi.org/10.29252/ijree.3.2.115 https://doi.org/10.29252/ijree.3.2.115 https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 53 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) Ahmed, R., Al-kadi, A., & Hagar, T. (2020). Enhancements and limitations to ICT-based informal language learning: Emerging research and opportunities (R. Ahmed, A. Al-kadi, & T. Hagar (Eds.)). IGI Global. https://doi.org/10.4018/978-1-7998-2116-8 Alelaimat, A. M., Ihmeideh, F. M., & Alkhawaldeh, M. F. (2020). Preparing preservice teachers for technology and digital media integration: Implications for early childhood teacher education programs. International Journal of Early Childhood, 52(3), 299–317. https://doi.org/10.1007/s13158-020-00276-2 Alkamel, M. A. A., & Chouthaiwale, S. S. (2018). The use of ICT tools in English language teaching and learning: A literature review. Veda’s Journal of English Language and Literature- JOELL, 5(2), 29–33. http://joell.in/wp-content/uploads/2018/04/29-33-THE-USE-OF- ICT-TOOLS-IN-ENGLISH-LANGUAGE.pdf Amir, Z., Ismail, K., & Hussin, S. (2011). Blogs in language learning: Maximizing students‟ collaborative writing. Procedia - Social and Behavioral Sciences, 18, 537–543. https://doi.org/10.1016/j.sbspro.2011.05.079 Antonio, A., & Tuffley, D. (2014). The gender digital divide in developing countries. Future Internet, 6(4), 673–687. https://doi.org/10.3390/fi6040673 Blake, R. (2013). Brave new digital classroom: Technology and foreign language learning. Georgetown University Press. Brown, H. D. (2000). Principles of language learning and teaching. Longman. Bryan, V. C., & Wang, V. X. (Eds.). (2013). Technology use and research approaches for community education and professional development. IGI Global. https://doi.org/10.4018/978-1-4666-2955-4 Chai, C. S., Koh, J. H. L., & Tsai, C.-C. (2010). Facilitating preservice teachers‟ development of technological, pedagogical, and content knowledge (TPACK). Journal of Educational Technology & Society, 13(4), 63–73. https://www.learntechlib.org/p/52307/ Ciptaningrum, D. S. (2017). The development of the survey of technology use, teaching, and technology-related learning experiences among pre-service English language teachers in Indonesia. Journal of Foreign Languange Teaching and Learning, 2(2), 11–26. https://doi.org/10.18196/ftl.2220 Ciptaningrum, D. S. (2018). The story of “Julie”: A life history study of the learning experiences of an Indonesian English language teacher in implementing ICT in her classroom. In S. Madya, F. Hamied, W. A. Renandya, C. Coombe, & Y. Basthomi (Eds.), ELT in Asia in the digital era: Global citizenship and identity - Proceedings of the 15th Asia TEFL and 64th TEFLIN international conference on English language teaching (pp. 495–503). Routledge. https://www.taylorfrancis.com/chapters/edit/10.1201/9781351217064-71/story-julie-life- history-study-learning-experiences-indonesian-english-language-teacher-implementing-ict- classroom-ciptaningrum Deng, F., Chai, C. S., Tsai, C.-C., & Lee, M.-H. (2014). The relationships among Chinese practicing teachers‟ epistemic beliefs, pedagogical beliefs and their beliefs about the use of ICT. Journal of Educational Technology & Society, 17(2), 245–256. Ding, A.-C. E., Ottenbreit-Leftwich, A., Lu, Y.-H., & Glazewski, K. (2019). EFL teachers‟ pedagogical beliefs and practices with regard to using technology. Journal of Digital Learning in Teacher Education, 35(1), 20–39. https://doi.org/10.1080/21532974.2018.1537816 Ersanli, C. Y. (2016). Improving technological pedagogical content knowledge (TPACK) of pre- service English language teachers. International Education Studies, 9(5), 18–27. https://doi.org/10.5539/ies.v9n5p18 https://doi.org/10.4018/978-1-7998-2116-8 https://doi.org/10.1007/s13158-020-00276-2 http://joell.in/wp-content/uploads/2018/04/29-33-THE-USE-OF-ICT-TOOLS-IN-ENGLISH-LANGUAGE.pdf http://joell.in/wp-content/uploads/2018/04/29-33-THE-USE-OF-ICT-TOOLS-IN-ENGLISH-LANGUAGE.pdf https://doi.org/10.1016/j.sbspro.2011.05.079 https://doi.org/10.3390/fi6040673 https://doi.org/10.4018/978-1-4666-2955-4 https://www.learntechlib.org/p/52307/ https://doi.org/10.18196/ftl.2220 https://www.taylorfrancis.com/chapters/edit/10.1201/9781351217064-71/story-julie-life-history-study-learning-experiences-indonesian-english-language-teacher-implementing-ict-classroom-ciptaningrum https://www.taylorfrancis.com/chapters/edit/10.1201/9781351217064-71/story-julie-life-history-study-learning-experiences-indonesian-english-language-teacher-implementing-ict-classroom-ciptaningrum https://www.taylorfrancis.com/chapters/edit/10.1201/9781351217064-71/story-julie-life-history-study-learning-experiences-indonesian-english-language-teacher-implementing-ict-classroom-ciptaningrum https://doi.org/10.1080/21532974.2018.1537816 https://doi.org/10.5539/ies.v9n5p18 https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 54 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) Golonka, E. M., Bowles, A. R., Frank, V. M., Richardson, D. L., & Freynik, S. (2014). Technologies for foreign language learning: A review of technology types and their effectiveness. Computer Assisted Language Learning, 27(1), 70–105. https://doi.org/10.1080/09588221.2012.700315 Gönen, S. İ. K. (2019). A qualitative study on a situated experience of technology integration: Reflections from pre-service teachers and students. Computer Assisted Language Learning, 32(3), 163–189. https://doi.org/10.1080/09588221.2018.1552974 Goodwin, J. (1991). Teaching pronunciation. In M. Celce-Murcia (Ed.), Teaching English as a second or foreign language (pp. 117–138). Heinle & Heinle - Thomson Learning. Hidayati, T. (2016). Integrating ICT in English language teaching and learning in Indonesia. JEELS (Journal of English Education and Linguistics Studies), 3(1). https://doi.org/10.30762/jeels.v3i1.173 Hutchison, A., & Reinking, D. (2011). Teachers‟ perceptions of integrating information and communication technologies into literacy instruction: A national survey in the United States. Reading Research Quarterly, 46(4), 312–333. https://doi.org/10.1002/RRQ.002 Kartchava, E., & Chung, S. (2015). Pre-service and in-service English as a second language teachers‟ beliefs about the use of digital technology in the classroom. Studies in English Language Teaching, 3(4), 355–383. https://doi.org/10.22158/selt.v3n4p355 Keengwe, J. (Ed.). (2013). Research perspectives and best practices in educational technology integration. IGI Global. https://doi.org/10.4018/978-1-4666-2988-2 Kern, R., Ware, P., & Warschauer, M. (2017). Network-based language teaching. In Second and foreign language education (pp. 197–209). Springer International Publishing. https://doi.org/10.1007/978-3-319-02246-8_30 Kılıçkaya, F., & Krajka, J. (2012). Can the use of web-based comic strip creation tool facilitate EFL learners‟ grammar and sentence writing? British Journal of Educational Technology, 43(6), E161–E165. https://doi.org/10.1111/j.1467-8535.2012.01298.x Koehler, M. J., Mishra, P., Kereluik, K., Shin, T. S., & Graham, C. R. (2014). The technological pedagogical content knowledge framework. In Handbook of research on educational communications and technology (pp. 101–111). Springer New York. https://doi.org/10.1007/978-1-4614-3185-5_9 Koehler, M. J., Mishra, P., & Yahya, K. (2007). Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy and technology. Computers & Education, 49(3), 740–762. https://doi.org/10.1016/j.compedu.2005.11.012 Koh, J. H. L., Chai, C. S., Benjamin, W., & Hong, H.-Y. (2015). Technological pedagogical content knowledge (TPACK) and design thinking: A framework to support ICT lesson design for 21st century learning. The Asia-Pacific Education Researcher, 24(3), 535–543. https://doi.org/10.1007/s40299-015-0237-2 Krashen, S. D. (1985). The input hypothesis: Issues and implications. Addison-Wesley Longman Ltd. Kurt, G., Akyel, A., Koçoğlu, Z., & Mishra, P. (2014). TPACK in practice: A qualitative study on technology integrated lesson planning and implementation of Turkish pre-service teachers of English. ELT Research Journal, 3(3), 153–166. https://dergipark.org.tr/tr/download/article-file/63644 Kwangsawad, T. (2016). Examining EFL pre-service teachers‟ TPACK trough self-report, lesson plans and actual practice. Journal of Education and Learning (EduLearn), 10(2), 103–108. https://doi.org/10.11591/edulearn.v10i2.3575 https://doi.org/10.1080/09588221.2012.700315 https://doi.org/10.1080/09588221.2018.1552974 https://doi.org/10.30762/jeels.v3i1.173 https://doi.org/10.1002/RRQ.002 https://doi.org/10.22158/selt.v3n4p355 https://doi.org/10.4018/978-1-4666-2988-2 https://doi.org/10.1007/978-3-319-02246-8_30 https://doi.org/10.1111/j.1467-8535.2012.01298.x https://doi.org/10.1007/978-1-4614-3185-5_9 https://doi.org/10.1016/j.compedu.2005.11.012 https://doi.org/10.1007/s40299-015-0237-2 https://dergipark.org.tr/tr/download/article-file/63644 https://doi.org/10.11591/edulearn.v10i2.3575 https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 55 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) Lantolf, J. P. (1994). Sociocultural theory and second language learning: Introduction to the special issue. The Modern Language Journal, 78(4), 418–420. https://doi.org/10.2307/328580 Liu, M., Moore, Z., Graham, L., & Lee, S. (2002). A look at the research on computer-based technology use in second language learning. Journal of Research on Technology in Education, 34(3), 250–273. https://doi.org/10.1080/15391523.2002.10782348 Majumdar, S. (2015). Emerging trends in ICT for education & training. Gen. Asia Pacific Reg. IVETA. https://www.stthomascollegebhilai.in/wp- content/uploads/2016/10/emergingtrendsinictforeducationandtraining.pdf Mei, B., Brown, G. T. L., & Teo, T. (2018). Toward an understanding of preservice English as a foreign language teachers‟ acceptance of computer-assisted language learning 2.0 in the People‟s Republic of China. Journal of Educational Computing Research, 56(1), 74–104. https://doi.org/10.1177/0735633117700144 Mirzaei, A., Rahimi Domakani, M., & Rahimi, S. (2016). Computerized lexis-based instruction in EFL classrooms: Using multi-purpose LexisBOARD to teach L2 vocabulary. ReCALL, 28(1), 22–43. https://doi.org/10.1017/S0958344015000129 Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. https://doi.org/10.1111/j.1467-9620.2006.00684.x Mishra, P., & Koehler, M. J. (2008). Introducing technological pedagogical content knowledge. Annual Meeting of the American Educational Research Association - Research on Schools, Neighborhoods, and Communities: Toward Civic Responsibility, 1–15. Öz, H. (2015). Assessing pre-service English as a foreign language teachers‟ technological pedagogical content knowledge. International Education Studies, 8(5), 119–130. https://doi.org/10.5539/ies.v8n5p119 Pica, T. (1994). Research on negotiation: What does it reveal about second-language learning conditions, processes, and outcomes? Language Learning, 44(3), 493–527. https://doi.org/10.1111/j.1467-1770.1994.tb01115.x Raygan, A., & Moradkhani, S. (2020). Factors influencing technology integration in an EFL context: Investigating EFL teachers‟ attitudes, TPACK level, and educational climate. Computer Assisted Language Learning, 1–22. https://doi.org/10.1080/09588221.2020.1839106 Regulation of the Minister of National Education No. 16 of 2007 on the Teacher‟s Standard of Academic Qualification and Competence, (2007). Regulation of the Minister of National Education No. 41 of 2007 on the Process Standard for Primary and Secondary Educational Units, (2007). Regulation of the Minister of National Education No. 78 of 2009 on the Implementation of International-Standard School on Primary and Secondary Educational Levels, (2009). Rosetta Stone and Project Tomorrow. (2013). World language teachers and their use of technology. Rosetta Stone. https://resources.rosettastone.com/assets/lp/9999999999/resources/speaking-the- language-of-the-21st-century.pdf Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189X015002004 Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–23. https://doi.org/10.17763/haer.57.1.j463w79r56455411 https://doi.org/10.2307/328580 https://doi.org/10.1080/15391523.2002.10782348 https://www.stthomascollegebhilai.in/wp-content/uploads/2016/10/emergingtrendsinictforeducationandtraining.pdf https://www.stthomascollegebhilai.in/wp-content/uploads/2016/10/emergingtrendsinictforeducationandtraining.pdf https://doi.org/10.1177/0735633117700144 https://doi.org/10.1017/S0958344015000129 https://doi.org/10.1111/j.1467-9620.2006.00684.x https://doi.org/10.5539/ies.v8n5p119 https://doi.org/10.1111/j.1467-1770.1994.tb01115.x https://doi.org/10.1080/09588221.2020.1839106 https://resources.rosettastone.com/assets/lp/9999999999/resources/speaking-the-language-of-the-21st-century.pdf https://resources.rosettastone.com/assets/lp/9999999999/resources/speaking-the-language-of-the-21st-century.pdf https://doi.org/10.3102/0013189X015002004 https://doi.org/10.17763/haer.57.1.j463w79r56455411 https://doi.org/10.21831/reid.v7i1.30521 Dyah S. Ciptaningrum, Nur Hidayanto P. S. Putro, Nila Kurnia Sari, & Nurqadriyanti Hasanuddin Page 56 - Copyright © 2021, REiD (Research and Evaluation in Education), 7(1), 2021 ISSN: 2460-6995 (Online) Swain, M., & Lapkin, S. (1995). Problems in output and the cognitive processes they generate: A step towards second language learning. Applied Linguistics, 16(3), 371–391. https://doi.org/10.1093/applin/16.3.371 Taopan, L. L., Drajati, N. A., & Sumardi, S. (2020). TPACK framework: Challenges and opportunities in EFL classrooms. Research and Innovation in Language Learning, 3(1), 1–22. https://doi.org/10.33603/rill.v3i1.2763 Tondeur, J., Roblin, N. P., van Braak, J., Fisser, P., & Voogt, J. (2013). Technological pedagogical content knowledge in teacher education: In search of a new curriculum. Educational Studies, 39(2), 239–243. https://doi.org/10.1080/03055698.2012.713548 Tour, E. (2015). Digital mindsets: Teachers‟ technology use in personal life and teaching. Language Learning & Technology, 19(3), 124–139. https://www.lltjournal.org/item/2923 Tseng, J.-J. (2016). Developing an instrument for assessing technological pedagogical content knowledge as perceived by EFL students. Computer Assisted Language Learning, 29(2), 302– 315. https://doi.org/10.1080/09588221.2014.941369 Turgut, Y. (2017). Tracing preservice English language teachers‟ perceived TPACK in sophomore, junior, and senior levels. Cogent Education, 4(1), 1368612. https://doi.org/10.1080/2331186X.2017.1368612 van Olphen, M. (2008). World language teacher education and educational technology: A look into CK, PCK, and TPCK. Annual Meeting of the American Educational Research Association - Research on Schools, Neighborhoods, and Communities: Toward Civic Responsibility. Wang, Y., Chung, C. J., & Hattingh, E. (2014). A blending approach in technology integrated ESL writing instruction. In M. Searson & M. Ochoa (Eds.), Proceedings of SITE 2014--Society for Information Technology & Teacher Education International Conference (pp. 1139–1144). Association for the Advancement of Computing in Education (AACE). https://www.learntechlib.org/primary/p/130922/ Warschauer, M. (1997). Computer-mediated collaborative learning: Theory and practice. The Modern Language Journal, 81(4), 470–481. https://doi.org/10.2307/328890 Yet, T. S., & Noordin, N. B. (2017). The use of ICT among pre-service English language teachers. International Journal of English Language Education, 5(1), 100–112. https://doi.org/10.5296/ijele.v5i1.10779 https://doi.org/10.1093/applin/16.3.371 https://doi.org/10.33603/rill.v3i1.2763 https://doi.org/10.1080/03055698.2012.713548 https://www.lltjournal.org/item/2923 https://doi.org/10.1080/09588221.2014.941369 https://doi.org/10.1080/2331186X.2017.1368612 https://www.learntechlib.org/primary/p/130922/ https://doi.org/10.2307/328890 https://doi.org/10.5296/ijele.v5i1.10779