Australasian Journal of Educational Technology, 2020, 36(2). 1 Assessing the digital competence of educators in social studies: An analysis in initial teacher training using the TPACK- 21 model Diego Miguel-Revilla, José María Martínez-Ferreira, María Sánchez-Agustí University of Valladolid, Spain Fostering the digital competence of educators is a key aspect that can be addressed in initial teacher training. The TPACK model (Mishra & Koehler, 2006) can be complemented with an approach that takes the Digital Competence of Educators framework (Redecker & Punie, 2017) and 21st century competences into account. This study analysed the practical utility of this conceptual model, and the effectiveness of a teaching intervention in a university setting, during two academic years, with social studies secondary education prospective teachers. Using a quantitative approach and the TPACK-21 questionnaire (Valtonen et al., 2017), the study examined the starting point and evolution of the participants. Results indicate the adequacy of the framework and instrument used and favourable progress towards competence after the assessment of seven factors. Although pre-service teachers showed a low degree of confidence regarding their capabilities of integrating technological with pedagogical and content knowledge, especially in three of the components, this obstacle was overcome after the intervention. It is possible to conclude that the pedagogical and conceptual orientation of the teaching proposal has shown a positive effect, evidencing the effectiveness of a comprehensive approach capable of adapting to the specificity and challenges of social studies education. Implications for practice or policy: • Pre-service teachers can benefit from an approach that integrates the TPACK model in social studies education. • Instructors can inform their teaching practice by incorporating the Digital Competence of Educators framework (Redecker & Punie, 2017) and 21st century competences in initial teacher training. • Teacher educators can provide prospective teachers with tools that can help develop their technological pedagogical content knowledge. Keywords: educational technology, TPACK, digital competence of educators, initial teacher training, social studies education, ICT Introduction and literature review The use of technology in educational contexts has, for quite some time, stopped being a mere choice and instead become a reality (Zhao & Lei, 2009) and something that should be adequately addressed in initial teacher training. This requires adapting to the new educational challenges that prospective teachers will face in the future and addressing those difficulties that they usually find in their preparation (Gutiérrez- Martín, Palacios-Picos, & Torrego-Egido, 2010). From this point of view, the digital competence of educators has been established in the last few years as a comprehensive concept that should be discussed if what is intended is to improve teacher education. This aspect has recently been synthetised in the European Union with the European Framework for the Digital Competence of Educators (DigCompEdu) (Redecker & Punie, 2017). The framework takes into account different areas related to teaching practices to establish, in each of them, a series of specific competences. In this model – an adaptation for educational contexts of the Digital Competence Framework for Citizens (Carretero, Vuorikari, & Punie, 2017) – four key pedagogic competences have been established: the use of digital resources, everything related with teaching and learning processes, assessment and feedback, and last of all, strategies to give students a voice. These are supplemented with a specific competence based on professional engagement and with another one focused on the fostering of the digital competence of learners. Australasian Journal of Educational Technology, 2020, 36(2). 2 DigCompEdu (Redecker & Punie, 2017) evidences that simply instructing pre-service teachers how to use digital tools and resources might not be enough, and that they should be capable of going beyond. Other organisms and institutions, including the United Nations Educational, Scientific and Cultural Organization, the Organisation for Economic Co-operation and Development, and the European Commission, have also called for the integration of a competence-based framework in the educational world (Halász & Michel, 2011) and for the development of the 21st century competences. These include the need for critical thinking and the ability to solve problems in a reflective, collaborative, and creative way (Mishra & Mehta, 2017; Voogt & Roblin, 2012). From this point of view, digital competence should be able to interact with other competences of diverse origins that also address the challenges facing our society. The search for a framework capable of assessing the digital competence of educators has been a necessity. The TPACK model has provided, since its conceptualisation, a way to appropriately integrate all aspects that come into play once technologies are introduced in educational processes (Mishra & Koehler, 2006). Following the idea initially advanced by Shulman (1986), pedagogical knowledge (PK) can be integrated with content knowledge (CK) in order to give shape to pedagogical content knowledge (PCK), an intersection that avoids the consideration of both elements as exclusive and that describes the necessary knowledge to teach each discipline. By introducing a new element capable of interacting with the other two, linked with technological knowledge (TK) on this occasion, it is possible to obtain two new factors: technological content knowledge (TCK) and pedagogical content knowledge (TPK). While the first one addresses the way in which technologies can be useful to represent contents, the second one focuses on the way the digital world can be applied for educational purposes (Bower, 2017). As a whole, TPACK becomes a useful framework capable of describing the complex interaction of technological, pedagogical and content knowledge, encompassing these three areas in a simultaneous and comprehensive way (Koehler & Mishra, 2009). Mishra and Koehler’s (2006) TPACK model is useful as a theoretical conceptualisation capable of substantiating an adoption of the digital competence of trainee teachers (in both initial and permanent lifelong training), but also as a useful assessment tool (Cabero & Barroso, 2016). The assessment proposals derived from this framework are numerous and not necessarily homogeneous, prioritising certain factors in order to develop specific questionnaires and other instruments (Chai, Koh, & Tsai, 2016). Previous initiatives focusing on the assessment of prospective teachers’ self-perception (Schmidt et al., 2009) have evolved or have been adapted to examine, for instance, a constructivist pedagogy orientation, like the C- TPACK questionnaire (Koh, Chai, & Tsai, 2014) or the use of specific technological instruments, such as TPCK-W (Lee & Tsai, 2010). At the same time, the need to address the fostering of 21st century competences or skills has resulted in their conceptual integration with this model due to the creation of instruments such as the TPACK-21 questionnaire (Valtonen et al., 2017), which has been used in this study. Although the empirical research based on Mishra and Koehler’s (2006) TPACK model has analysed different disciplines, including experimental sciences (Deng, Chai, So, Qian, & Chen, 2017; Jang & Tsai, 2013) or mathematics (Karatas, Piskin Tunc, Yilmaz, & Karaci, 2017), social studies has only been examined by less than 3% of recent studies (Willermark, 2018). This is something that has also affected the assessment of the integration of the digital competence of educators in initial teacher training. Regardless, research reflects the potential and practical use of TPACK for improving the formative process of prospective social studies teachers due to the application of proposals capable of being of use as the basis for the design of educational interventions (Cózar-Gutiérrez, Zagalaz, & Sáez-López, 2015; Gómez- Trigueros, 2015). Mishra and Koehler’s (2006) TPACK model also provides an important analytical potential, examining teaching practices related to social studies (Curry & Cherner, 2016; Van Vaerenewyck, Shinas, & Steckel, 2017) or assessing impediments or difficulties specific to social studies education and the promotion of digital competence during formative interventions (Gómez-Trigueros, 2016). At the same time, the use of this framework has enabled the assessment of digital competence level among social studies trainee teachers (Colomer-Rubio, Sáiz-Serrano, & Bel-Martínez, 2018) and the establishment of a series of comparative studies related to the degree of effectivity of particular proposals. The analysis of these cases, specifically designed for the development of the digital competence of educators, has evidenced that progress towards competence among participants is possible (Beriswill, Bracey, Sherman-Morris, Huang, & Lee, 2016). Australasian Journal of Educational Technology, 2020, 36(2). 3 Purpose of the study This study aimed to examine whether the TPACK model, and specifically a version based on 21st century competences, is a coherent and useful assessment tool for analysing the acquisition of digital competence in social studies initial teacher training. Simultaneously, the main objective of the research was assessing the effectiveness of a formative intervention based on the principles of that model (TPACK-21) (Valtonen et al., 2017) and the DigCompEdu framework (Redecker & Punie, 2017). This intervention was applied with an entire sample of social studies pre-service teachers during two academic years with two subsamples of yearly cohorts, which were compared to examine potential differences between them before the intervention. In line with the main purpose of this study, it was expected that the conceptual and pedagogical orientation of this intervention would be capable of facilitating the acquisition of the digital competence of prospective teachers. The research questions that guided this study are the following: (1) Is the TPACK-21 questionnaire an adequate instrument to assess the digital competence of educators in social studies initial teacher training? (2) What is the self-perception of the digital competence of educators (before and after the formative intervention) of those secondary education prospective teachers that took part in the study? (3) What are the effects of an intervention that explicitly addresses the competences established in the DigCompEdu framework (Redecker & Punie, 2017) and that takes into account the specificities of this particular discipline? Methodological approach This study was based on a quantitative research design (Creswell, 2012). In particular, the study followed a one-group pretest-posttest quasi-experimental design, an approach that makes use of pretest and posttest observations (Shadish, Cook, & Campbell, 2002). A control group was not used due to the compulsory character of the course in which the intervention took place, something that has not enabled finding an equivalent group of participants. This analysis focused its attention on seven factors linked to the categories integrated in the TPACK model, including aspects related to disciplinary, pedagogical and technological knowledge, as well as interaction between all of them (Koehler & Mishra, 2009). Context and participants This research took place in two different academic years: 2017–2018 and 2018–2019. In both cases, participants were all students enrolled in the geography and history specialties of the Master’s Degree in Teaching in Secondary Schools in the University of Valladolid (Spain). The intervention took place during 3 months in both 2017 and 2018, for a total of 30 hours in each of the occasions, in the framework of a course focused on teaching innovation in history, geography, and art history. The intervention did not suffer from any changes between each of the two years: the content, course duration, and professors in charge of the course remained the same, as well as the materials and resources that were used. The study obtained information from a total of 50 trainee teachers, approximately equally distributed between men and women, and with a mean age of 25.4 years (SD = 5.3). Participants belonged to two cohorts: although 27 prospective teachers participated in the first academic year, 23 more were engaged a year later. The decrease in the number of participants between both academic years is the result of the circumstances derived from the students enrolled annually in the master’s degree. From this point of view, a non- probabilistic sampling was used, justifying the selection of the participants according to accessibility criteria (Neuman, 2007) and focusing attention on the results of a specific intervention. A convenience strategy that can also be characterised as typical was used (Wellington, 2015). Participants shared the main characteristics and features of those students who usually enrol in this master’s degree, both in age and academic background. Australasian Journal of Educational Technology, 2020, 36(2). 4 Instrument, data processing, and analysis In order to obtain information on the degree of competence perceived by the participants in relation to each of the categories of analysis, the TPACK-21 questionnaire was used (Valtonen et al., 2017). On this occasion, disciplinary content knowledge was approached from the framework of social studies education. The questionnaire, developed by a team of researchers of Finnish and American origin, and whose psychometric properties have been assessed, tries to adapt the TPACK model to 21st century competences. The questionnaire was translated into the Spanish language for this study using experts provided by the official translation services of our institution, although a back-translation procedure was not used. The questionnaire is composed of 38 statements that are codified using a 6-point Likert-like scale, with values ranging from 1 to 6 (1 = I need a lot of additional knowledge about the topic; 6 = I have strong knowledge about the topic). The items are linked with seven scales, including PK (7 items in total, such as “I can guide students in planning their own learning”), TK (4 items, such as “I am familiar with new technologies and their features”), CK (another 4 items, including “I am familiar with recent research in social studies”), PCK (6 items, such as “In social studies, I know how to guide students’ critical thinking”), TPK (6 items, including “I know how to use ICT in teaching as a tool for students’ creative thinking”), TCK (4 items, such as “I know ICT-applications which are used by professionals in social studies”), and TPACK (with 7 items, including “In teaching social studies, I know how to use ICT as a tool for students to plan their own learning”). The instrument was made available by its original authors (Valtonen et al., 2017) and is accessible via AJET (https://doi.org/10.14742/ajet.3518). With the aim of assessing the progress shown by the participants, the same questionnaire was applied before and after the intervention with the informed consent of the pre-service teachers. The paired responses were transferred to the SPSS software to process the different variables. After calculating the results for each of the seven factors (PK, TK, CK, PCK, TPK, TCK, and TPACK), internal consistency was measured. Cronbach’s α was analysed using α = .70 as an acceptable threshold (Nunnally & Bernstein, 1994) and compared with studies that have also used this instrument in the past. A series of Pearson’s correlations between the seven factors was run with the information obtained before and after the intervention. An independent samples t test was used to find for potential differences between the two cohorts. After this procedure, a series of dependent or paired-samples t test was applied to all participants with the objective of establishing a comparison between the initial and final results. For all these tests, p < .05 was established as a significant threshold. Additionally, Cohen’s d values have also been provided in order to examine effect sizes, with d = .20 being interpreted as a small effect size, d = .40 as a medium effect size, and over d = .80 as a large effect size (Cohen, 1988), although context should be taken into account in educational research (Coe, 2017). Design of the intervention The formative proposal tried to avoid offering a mere theoretical proposition, applying instead a more active and practical teaching methodology based on a project-based learning approach. From the beginning of the 30-hour intervention, participants (who were grouped in small teams) started working on the conceptualisation, design, and implementation of a project, always within the framework of the course. Prospective teachers had to create an online platform based on the implementation of a learning management system such as Moodle or Docebo, a content management system like Drupal or WordPress, or a massive open online course architecture (using Open edX. By using a particular platform chosen and configured by them, participants had to design a pedagogical proposal capable of addressing historical, geographic or art history contents, and specifically prepared for a particular context. Prospective teachers integrated digital resources and tools in their platform, always with an educational aim in mind. They designed interactive activities by using H5P, made use of textual and audiovisual historical resources annotated with EDpuzzle, created and personalised interactive historical timelines with tools such as TimelineJS, and also embedded geolocation applications. Discussion forums, feedback chats, QR codes, forms, and other systems for student assessment were also used. The course designers established a series of incentives for prospective teachers, enabling them to introduce a methodological approach different from the traditional one, still persistent in social studies education. Faced with the need to transform the roles assumed by students and teachers, and to take advantage of the potential offered by the use of technologies, mailto:https://doi.org/10.14742/ajet.3518 Australasian Journal of Educational Technology, 2020, 36(2). 5 participants designed activities to promote more active and collaborative classroom practices that, due to the gradual introduction of gamified elements, could facilitate a higher degree of involvement on the part of students. The course designers focused on how to address curricular contents to foster understanding in social studies with the aim of encouraging an adequate interaction between disciplinary contents and pedagogical and technological factors. Each group addressed some of the more prominent learning difficulties among students, emphasising how to use digital resources in order to develop critical and historical thinking (Seixas, 2017; van Boxtel & van Drie, 2018). At the same time, participants also designed their proposals considering how to guide students in order to avoid setting aside the social or public dimension of this discipline, making them aware of the relationship between the past and the present (Miguel-Revilla & Sánchez-Agustí, 2018). Complementing the TPACK model, the intervention was guided by the DigCompEdu framework (Redecker & Punie, 2017), highlighting in the classroom the four key areas linked to educators’ pedagogic competences: the use, creation and distribution of digital resources, the employment of digital tools in teaching and learning, assessment and feedback, and, last of all, learner empowerment due to the use of technology (Redecker & Punie, 2017). Participants worked on all these aspects, using and modifying digitised historical, geographic, and artistic sources, but also guiding students in the search for and analysis of information, always from a critical, reflective, creative, and collaborative point of view, according to the standards linked to 21st century competences. Results An examination of the TPACK-21 model First of all, the way the questionnaire was able to address each of the factors that are part of the theoretical framework of the TPACK model (Mishra & Koehler, 2006) was examined. The relationship between the different categories is considered with the aim of analysing their interaction in practical contexts. Attending, in the first place, to the internal consistency before and after the use of the TPACK-21 questionnaire (Valtonen et al., 2017) (Table 1), the Cronbach’s α coefficient consistently adopted high values (α between .85 and .94) for each of the factors except for CK (α = .75 and α = .73), although levels above .70 are usually deemed acceptable (Nunnally & Bernstein, 1994). Consistency levels are in line with results obtained after the application of the instrument by its original creators, in which each of the scales assumes α values between .88 and .95 (Valtonen, Kukkonen, Kontkanen, Mäkitalo-Siegl, & Sointu, 2018). Table 1 Pearson’s correlations and internal consistency for each of the factors (complete data set) Factor 1 2 3 4 5 6 7 1. PK --- -.03 .64*** .90*** .19 .16 .22 2. TK .20 --- .08 -.07 .70*** .66*** .70*** 3. CK .72*** .41** --- .68*** .31* .34* .35* 4. PCK .91*** .23 .72*** --- .24 .18 .27 5. TPK .48*** .85*** .58*** .47** --- .72*** .94*** 6. TCK .30* .87*** .42** .26 .86*** --- .74*** 7. TPACK .43** .86*** .55*** .43** .97*** .89*** --- Cronbach’s α (pre) Cronbach’s α (post) .85 .89 .88 .91 .75 .73 .89 .90 .93 .93 .90 .88 .93 .94 Note. Correlations corresponding to the initial test are located in the upper diagonal, while the ones corresponding to the final test can be found in the lower diagonal. *p < .05. **p < .01. ***p < .001. Secondly, after verifying different assumptions, a series of Shapiro-Wilk tests for normality were applied. Results are consistent with a normal distribution for all factors in the initial application: PK (W = .97, p = .18), TK (W = .98, p = .69), CK (W = .96, p = .11), PCK (W = .96, p = .113), TPK (W = .97, p = .34), TCK (W = .97, p = .19), and TPACK (W = .98, p = .50). Additionally, histograms were examined for each factor, Australasian Journal of Educational Technology, 2020, 36(2). 6 as well as skewness (with values ranging between -.581 and .401) and kurtosis (with values ranging between -.362 and .623). Equivalent results were obtained after the intervention. A series of Pearson’s correlations among the different factors were run, analysing the information obtained before and after the end of the intervention with the objective of assessing the conceptual structure of the TPACK-21 questionnaire (Valtonen et al., 2017) and the TPACK model (Mishra & Koehler, 2006). The results obtained before and after the intervention evidence a strong positive correlation, which is statistically significant, among a series of different factors. Therefore, TK shows a correlation in the initial questionnaire with TPK (r = .70, p < .001), TCK (r = .66, p < .001), and TPACK (r = .70, p < .001), that is, with those factors in which technology assumes a key role, as expected. This can also be observed in the information obtained after the intervention: TPK (r = .85, p < .001), TCK (r = .87, p < .001), and TPACK (r = .86, p < .001). Regarding CK, the data shows a positive correlation, also statistically significant, with those factors in which content plays an important part. This can be corroborated in the initial application of the instrument: PCK (r = .68, p < .001), TCK (r = .34, p = .02), and TPACK (r = .35, p = .01), but also after the intervention: PCK (r = .72, p < .001), TCK (r = .42, p < .001), and TPACK (r = .55, p = .002). The same happened with PK, although on this occasion, there is only a significant positive correlation with PCK (r = .90, p < .001) in the initial application, and with PCK (r = .91, p < .001), TPK (r = .48, p < .001) and TPACK (r = .43, p = .002) in the final one. Additionally, a positive and statistically significant correlation can be observed between TPACK and four other factors in the initial application – TK (r = .70, p < .001), CK (r = .35, p = .01), TPK (r = .94, p < .001), and TCK (r = .74, p < .001) – and with the other six factors after the intervention: PK (r = .43, p = .002), TK (r = .86, p < .001), CK (r = .55, p < .001), PCK (r = .43, p = .002), TPK (r = .97, p < .001), and TCK (r = .89, p < .001), in consonance with the model. This is especially strong among the variables in which the technological factor assumes a noticeable role (TK, TPK, and TCK). Starting point and progress after the intervention Focusing, on this occasion, on the results that were obtained before and after the intervention, a comparative was established regarding the evolution that was detected among participants using a series of dependent or paired-samples t tests. These were applied to each of the seven factors that were analysed, with the intention of finding differences between the categories. Cohen’s d values have also been provided for each analysis. Before presenting the results of the entire sample, and in order to establish whether significant differences can be detected between the two cohorts, independent samples t tests were run for each of the seven factors of the TPACK model. Focusing exclusively on the results obtained before the intervention, the analysis shows that there are no statistical differences for any of the components: PK (t(48) = -1.33, p = .12, d = .39), TK (t(48) = .272, p = .79, d = .08), CK (t(48) = -.66, p = .51, d = .18), PCK (t(48) = -1.01, p = .28, d = .32), TPK (t(48) = 1.18, p = .25, d = .34), TCK (t(48) = -.39, p = .70, d = .11), and TPACK (t(48) = 1.58, p = .12, d = .45). After verifying that both cohorts are comparable, a series of dependent or paired-samples were applied for the complete set of responses (Table 2). Table 2 Results of the dependent samples t tests, comparing data obtained before and after the intervention Factor Instrument application Mean difference 95% CI mean difference Initial test Final test M SD M SD d t PK 4.31 .73 4.60 .68 -.29 [-.49, -.08] .41 -2.81** TK 3.85 1.17 4.68 .93 -.82 [-1.05, -.59] .79 -7.25*** CK 3.93 .78 4.40 .71 -.47 [-.69, -.25] .63 -4.24*** PCK 4.21 .84 4.52 .69 -.31 [-.51, -.11] .40 -3.09** TPK 3.22 1.00 4.43 .92 -1.20 [-1.43, -.97] 1.26 -10.46*** TCK 3.09 1.25 4.81 .81 -1.72 [-2.04, -1.39] 1.63 -10.65*** TPACK 3.12 1.05 4.48 .87 -1.36 [-1.57, -1.14] 1.41 -12.57*** Note. n = 50; df = 49; CI = confidence interval; M = mean; SD = standard deviation. *p < .05. **p < .01. ***p < .001. Australasian Journal of Educational Technology, 2020, 36(2). 7 The global analysis of the results offers an overall picture in which it is possible to detect positive progress. The dependent t tests evidence a significant difference and medium effect sizes between the initial and the final application of the questionnaire for each of the seven factors, although this progress is more prominent in relation to some categories. Examining the first three – PK (t(49) = -2.81, p = .007, d = .41); TK (t(49) = -7.25, p < .001, d = .79); and CK (t(49) = -4.24, p < .001, d = .63) – it is possible to detect a relatively high starting point (M = 4.31, M = 3.85 and M = 3.93, respectively), at least in relation to TPK, TCK, and TPACK, although not with PCK. Nonetheless, PK, TK, and CK evidence clear progress towards significantly higher values (M = 4.60, M = 4.68, and M = 4.40, respectively). Regarding the four other factors, the comparisons signal a positive and statistically significant progress among prospective teachers: PCK (t(49) = -3.09, p = .003, d = .40); TPK (t(49) = -10.46, p < .001, d = 1.26); and TCK (t(49) = -10.65, p < .001, d = 1.63). Effect size can be considered medium for PCK, but quite large for TPK and TCK. It is relevant to highlight the starting point of the participants because, putting aside the evolution of PCK (M = 4.21 at the beginning, in contrast with M = 4.52 after the intervention), both TPK and TCK (linked to the use of technology) show a lower starting point than the rest of the categories (M = 3.22 and M = 3.09, respectively). This can also be detected in relation to the TPACK factor (M = 3.12 before the intervention), although it is relevant to note that a significant progress and a considerably large effect size can also be found in this category (t(49) = -12.57, p < .001, d = 1.41). On all these occasions, the average values found after the intervention are very high, with a higher contrast between the starting and final position of the participants (M = 4.43, M = 4.81 and M = 4.48 in the final test for TPK, TCK, and TPACK, respectively). Moreover, this comparison also indicates that the two main categories linked to pedagogical content (PK and PCK) already show relatively high initial values (M = 4.31 and M = 4.21), while factors such as TPACK have an inferior starting point (M = 3.12), displaying a clearer statistically significant evolution after the intervention. Discussion and conclusions This study examined the usefulness of the TPACK model (Mishra & Koehler, 2006) in initial teacher training, making use of the TPACK-21 questionnaire (Valtonen et al., 2017) to assess the digital competence of educators in social studies pre-service teachers. Additionally, the effects of an intervention based on the DigCompEdu framework (Redecker & Punie, 2017) and 21st century competences were analysed, finding a positive progress in all seven TPACK factors. Consequently, it is necessary to discuss aspects related to the instrument that was used, the results that were obtained before and after the intervention, and the theoretical framework in which this research was framed. First of all, the analysis evidences that the TPACK-21 questionnaire (Valtonen et al., 2017) is a valuable tool for social studies initial teacher training, demonstrating a correspondence with the conceptualisation of the digital competence of educators that has been adopted. The questionnaire has shown an adequate degree of internal consistency, in line with previous applications of the instrument (Valtonen et al., 2018). Alongside its coherent structure, the TPACK-21 model has made it possible to address how teacher education can promote different competence areas, in consonance with the planning and design of the intervention and with many of the key principles that guide social studies education (Seixas, 2017; van Boxtel & van Drie, 2018). Secondly, the progress towards competence that was found after establishing a comparison between the results that were obtained before and after the intervention signal the effectiveness of the intervention in order to promote an improvement in the self-perception of prospective teachers. This progress is consistent with the results found in previous studies that made use of the TPACK model in the framework of social studies teacher training, with a similar evolution for the seven factors that were analysed (Beriswill et al., 2016). At the same time, this points to the potential of teaching practices, which can affect all TPACK components in a comprehensive way (Gómez-Trigueros, 2015). From this point of view, the use of this theoretical framework can inform and orient teacher training, showing its utility to improve social studies trainee teachers’ capabilities during their formative phase (Cózar-Gutiérrez et al., 2015). As with this research, comparable studies have also revealed that PK and PCK factors (linked to the pedagogical component of TPACK) can show a less significant progress, starting from relatively higher levels (Beriswill et al., 2016). On the other hand, the results that were obtained in relation to the starting Australasian Journal of Educational Technology, 2020, 36(2). 8 point of the participants contrast with the data obtained by Colomer-Rubio et al. (2018) in a similar context, with prospective teachers evidencing more difficulties in CK and PCK (factors linked to disciplinary content knowledge). This is something that might be explained due to the characteristics of the participants in that study (undergraduate primary education trainee teachers), in contrast with the secondary education prospective teachers who took part in this study, with a specific disciplinary background in social studies. Regardless of the level of progress, the starting point of the participants in this study signals that those factors related to the technological components of the model (TK, TPK, and TCK, as well as TPACK) show relatively lower levels than the other ones. This is something than can also be observed in additional studies using the TPACK-21 questionnaire (Valtonen et al., 2018). The results of this research are also in in line with studies that have also analysed this aspect (Roig-Vila, Mengual-Andrés, & Quinto-Medrano, 2015). While some authors have criticised the potential to downgrade pedagogical and disciplinary content knowledge in favour of the technological component in teacher training (Cabero & Barroso, 2016), participants in this study initially showed difficulties with integrating purely technological elements with those linked to a pedagogical and content-based approach. With regard to the results, a lack of technological competence does not mean that initial teacher training should focus attention on TK exclusively; rather, the focus should be on TPK and TCK (the interaction of technological knowledge with pedagogical and content knowledge), in accordance with a comprehensive model (Bower, 2017). It is relevant to remember that, while the educational use of technology has been prevalent for a long time (Zhao & Lei, 2009), the DigCompEdu framework (Redecker & Punie, 2017) and other 21st century competence frameworks underscore the practical use of technology and the importance of combining technological, pedagogical and content knowledge to foster a more collaborative approach (Voogt & Roblin, 2012). Thirdly, it can be concluded that the TPACK model (Mishra & Koehler, 2006) and the DigCompEdu framework (Redecker & Punie, 2017) were useful as guides to help design the conceptual orientation of the initial teacher proposal that was applied on this particular occasion. Regarding TPACK, the conception advanced by Shulman (1986) seems to remain in full force. Just like sporadically supplementing disciplinary content knowledge with general notions of pedagogy might not be enough to adequately address teaching and learning, the same happens when it is time to integrate the digital or technological factor in teaching practices (Mishra & Koehler, 2006). At the same time, and understanding that the work on the digital competence of educators cannot be isolated from the necessities of our current context, this intervention tried to focus attention on the use of resources, student assessment, feedback, and the rest of pedagogical competences described in the DigCompEdu framework (Redecker & Punie, 2017). These aspects were accompanied, last of all, with a perspective capable of enhancing 21st century competences, assessing them in a comprehensive way due to the use of the TPACK-21 questionnaire (Valtonen et al., 2017). Again, underscoring how to foster critical thinking and collaborative, creative and reflective problem-solving skills (Mishra & Mehta, 2017) in initial teacher training can be a useful strategy for connecting not only pedagogical, but technological and content knowledge as well, as a way to address the challenges of today’s society while also considering the specificities of social studies education. In relation to policy and practice, results show the value of re-examining how the digital competence of educators is addressed in teacher training, and the relevance of stressing the influence of teacher education curriculum and its orientation. Studies focusing on initial teacher training have emphasised the positive relationship between the use of educational strategies based on reflective models and the promotion of interaction between technological, pedagogical and content knowledge (Baran, Canbazoglu-Bilici, Albayrak-Sari, & Tondeur, 2019). In this particular case, the potential of history and social studies to establish a clear relationship between the past and the present (Miguel-Revilla & Sánchez-Agustí, 2018), but also to allow the creation and use of multiple digitised historical resources, might have been useful to engage prospective teachers in their learning process. The practical utility of the intervention and the fact that a collaborative project-based learning methodology was used might have also been useful. From this point of view, allowing participants to experience an alternative approach to teacher training seems to have a positive effect that might inform practice. It is because of this, and due to the necessity of encouraging a more nuanced understanding of the digital competence of educators, that the use of a framework like DigCompEdu can become a resource of great value for orienting teacher training. This Australasian Journal of Educational Technology, 2020, 36(2). 9 study has showed that, while there is still a lot of work to be done, it is possible to design an intervention capable of adhering to these principles and useful for pre-service teachers. Limitations and future directions This study faced a series of limitations. On one hand, the intervention was only conducted with a total of 50 participants, a sample size that can be considered relatively small and only restricted to a specific context, potentially limiting the generalisation of the results. On the other hand, the aim of the study was not to simply analyse the self-perception of social studies pre-service teachers, but also to examine the effects of an intervention with a very specific pedagogical orientation. From this point of view, and while the research was limited by the number of prospective teachers enrolled annually in the Master’s Degree in Teaching in Secondary Schools, it was decided to conduct the intervention during two academic years, with two yearly cohorts. Additionally, although a pretest-posttest quasi-experimental design was used, the presence of a control group might have been useful for providing additional information. Another limitation has to do with the potential difficulties to replicate the intervention in other educational contexts due to the specificity of this particular course. Future studies might try to focus on this issue by establishing a protocol that could orient teaching practices and that may be replicated elsewhere. This might encompass more than one course or type of intervention, and could also be adapted to other disciplines, although taking into account the differences among them (Chai et al., 2016). Furthermore, an additional limitation is that no back-translation procedures were used to translate the TPACK-21 questionnaire into the Spanish language. Because the study was conducted in a context other than Finland, where the instrument was originally applied, it might be possible to investigate the psychometric properties of the questionnaire with a much larger sample and to conduct a confirmatory factor analysis to try to replicate the theoretical model. Future research may also consider applying a mixed-methods approach capable of supplementing the data obtained using the TPACK-21 questionnaire with one-to-one interviews. Despite the limitations, it is important to remember that there is progress being made in the assessment of teaching practices and educators’ initial training regarding their digital competence, including the design of questionnaires and other instruments linked to the TPACK model (Chai et al., 2016; Harris, Phillips, Koehler, Rosenberg, & Phillips, 2017). Future research directions might involve a bigger focus on many of the aspects that have been discussed in this study, including more emphasis on the examination of new instruments and theoretical frameworks, as well as how to adapt the specificities of social studies education in such models. For instance, it should be determined in the future whether alternative conceptualisations, such as the TPACK-21CQL model, which focuses on 21st century quality learning (Chai et al., 2019), are a better fit in the assessment of these competences. An analysis of domain-specific characteristics regarding social studies is among the elements that future research will have to focus on. Although research in this specific area is scarce, qualitative studies have started providing valuable information about how to address history or social studies in the classroom with the TPACK model in mind (Curry & Cherner, 2016; Van Vaerenewyck et al., 2017). In any case, there is still work to be done, and a comparative approach with other disciplines might be useful to determine if progress in all TPACK factors is equivalent or there are differences in other domains or contexts in initial teacher training. To sum up, there are many challenges that will need to be overcome in future research, including a lack of attention regarding social studies disciplines, a dependency on self-assessment on the part of teachers, and the influence of each particular educational context (Willermark, 2018), although advances in this field are encouraging for the future. Acknowledgments This work was supported by the Spanish Government’s Ministry of Economy, Industry and Competitiveness (MINECO) and co-financed by the European Regional Development Fund (ERDF), under Grant EDU2013-43782-P. Australasian Journal of Educational Technology, 2020, 36(2). 10 References Baran, E., Canbazoglu-Bilici, S., Albayrak-Sari, A., & Tondeur, J. (2019). Investigating the impact of teacher education strategies on preservice teachers’ TPACK. British Journal of Educational Technology, 50(1), 357–370. https://doi.org/10.1111/bjet.12565 Beriswill, J. E., Bracey, P. S., Sherman-Morris, K., Huang, K., & Lee, S. J. (2016). Professional development for promoting 21st century skills and Common Core state standards in foreign language and social studies classrooms. TechTrends, 60(1), 77–84. https://doi.org/10.1007/s11528-015-0004-5 Bower, M. (2017). The technology pedagogy and content knowledge (TPACK) framework and its implications. In M. Bower (Ed.), Design of technology-enhanced learning: Integrating research and practice (pp. 17–33). Bingley, UK: Emerald. https://doi.org/10.1108/9781787141827 Cabero, J., & Barroso, J. (2016). ICT teacher training: A view of the TPACK model / Formación del profesorado en TIC: una visión del modelo TPACK. Cultura y Educación, 28(3), 633–663. Carretero, S., Vuorikari, R., & Punie, Y. (2017). DigComp 2.1: The digital competence framework for citizens with eight proficiency levels and examples of use. Luxembourg: Publications Office of the European Union. https://doi.org/10.2760/38842 Chai, C.-S., Koh, J., & Teo, Y.-H. (2019). Enhancing and modeling teachers’ design beliefs and efficacy of technological pedagogical content knowledge for 21st century quality learning. Journal of Educational Computing Research, 57(2), 360–384. https://doi.org/10.1177/0735633117752453 Chai, C.-S., Koh, J., & Tsai, C.-C. (2016). A review of the quantitative measures of technological pedagogical content knowledge (TPACK). In M. C. Herring, M. J. Koehler, & P. Mishra (Eds.), Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 87– 106). New York, NY: Routledge. https://doi.org/10.4324/9781315771328 Coe, R. J. (2017). Effect size. In R. J. Coe, M. Waring, L. V. Hedges, & J. Arthur (Eds.), Research methods & methodologies in education (2nd ed., pp. 339–349). London, UK: Sage. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Colomer-Rubio, J. C., Sáiz-Serrano, J., & Bel-Martínez, J. C. (2018). Competencia digital en futuros docentes de ciencias sociales en educación primaria: análisis desde el modelo TPACK [Digital competence in future teachers of social sciences in primary education: Analysis of the TPACK framework]. Educatio Siglo XXI, 36(1), 107–128. https://doi.org/10.6018/j/324191 Cózar-Gutiérrez, R., Zagalaz, J., & Sáez-López, M. (2015). Creando contenidos curriculares digitales de ciencias sociales para educación primaria. Una experiencia TPACK para futuros docentes [Creating digital curricular contents of social sciences for primary education. A TPACK experience for future teachers]. Educatio Siglo XXI, 33(3), 147–168. https://doi.org/10.6018/j/240921 Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Boston, MA: Pearson. Curry, K., & Cherner, T. (2016). Social studies in the modern era: A case study of effective teachers’ use of literacy and technology. The Social Studies, 107(4), 123–136. https://doi.org/10.1080/00377996.2016.1146650 Deng, F., Chai, C.-S., So, H.-J., Qian, Y., & Chen, L. (2017). Examining the validity of the technological pedagogical content knowledge (TPACK) framework for preservice chemistry teachers. Australasian Journal of Educational Technology, 33(3), 1–14. https://doi.org/10.14742/ajet.3508 Gómez-Trigueros, I. M. (2015). El modelo TPACK en los estudios de grado para la formación inicial del profesorado en TIC [TPACK model in degree studies for initial teacher training in ICT]. Didáctica Geográfica, 22(16), 185–201. Retrieved from http://www.age- geografia.es/didacticageografica/index.php/didacticageografica/article/view/301 Gómez-Trigueros, I. M. (2016). La inclusión de las tecnologías en la formación inicial del profesorado: una intervención de aula a través del modelo TPACK [The inclusion of technology in initial teacher training: An intervention in the classroom with the TPACK model]. Tendencias Pedagógicas, 28, 133–152. https://doi.org/10.15366/tp2016.28.010 Gutiérrez-Martín, A., Palacios-Picos, A., & Torrego-Egido, L. (2010). La formación de los futuros maestros y la integración de las TIC en la educación: anatomía de un desencuentro [School teacher training and ICT integration in education: Anatomy of a mismatch]. Revista de Educación, 352, 267– 293. Retrieved from http://www.educacionyfp.gob.es/revista-de-educacion/numeros-revista- educacion/numeros-anteriores/2010/re353/re353-10.html Halász, G., & Michel, A. (2011). Key competences in Europe: Interpretation, policy formulation and implementation. European Journal of Education, 46(3), 289–306. https://doi.org/10.1111/j.1465- 3435.2011.01491.x https://doi.org/10.1111/bjet.12565 https://doi.org/10.1007/s11528-015-0004-5 https://doi.org/10.1108/9781787141827 https://doi.org/10.2760/38842 https://doi.org/10.1177/0735633117752453 https://doi.org/10.4324/9781315771328 https://doi.org/10.6018/j/324191 https://doi.org/10.6018/j/240921 https://doi.org/10.1080/00377996.2016.1146650 https://doi.org/10.14742/ajet.3508 http://www.age-geografia.es/didacticageografica/index.php/didacticageografica/article/view/301 http://www.age-geografia.es/didacticageografica/index.php/didacticageografica/article/view/301 https://doi.org/10.15366/tp2016.28.010 http://www.educacionyfp.gob.es/revista-de-educacion/numeros-revista-educacion/numeros-anteriores/2010/re353/re353-10.html http://www.educacionyfp.gob.es/revista-de-educacion/numeros-revista-educacion/numeros-anteriores/2010/re353/re353-10.html https://doi.org/10.1111/j.1465-3435.2011.01491.x https://doi.org/10.1111/j.1465-3435.2011.01491.x Australasian Journal of Educational Technology, 2020, 36(2). 11 Harris, J., Phillips, M., Koehler, M. J., Rosenberg, J., & Phillips, M. (2017). TPCK/TPACK research and development: Past, present, and future directions. Australasian Journal of Educational Technology, 33(3), i–viii. https://doi.org/10.14742/ajet.3907 Jang, S.-J., & Tsai, M.-F. (2013). Exploring the TPACK of Taiwanese secondary school science teachers using a new contextualized TPACK model. Australasian Journal of Educational Technology, 29(4), 566–580. https://doi.org/10.14742/ajet.282 Karatas, I., Piskin Tunc, M., Yilmaz, N., & Karaci, G. (2017). An investigation of technological pedagogical content knowledge, self-confidence, and perception of pre-service middle school mathematics teachers towards instructional technologies. Journal of Educational Technology & Society, 20(3), 122–132. Retrieved from https://www.jstor.org/stable/26196124 Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60–70. Retrieved from https://www.citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent- knowledge/ Koh, J., Chai, C.-S., & Tsai, C.-C. (2014). Demographic factors, TPACK constructs, and teachers’ perceptions of constructivist-oriented TPACK. Journal of Educational Technology & Society, 17(1), 185–196. Retrieved from https://www.j-ets.net/ETS/issues619c.html?id=62 Lee, M.-H., & Tsai, C.-C. (2010). Exploring teachers’ perceived self efficacy and technological pedagogical content knowledge with respect to educational use of the World Wide Web. Instructional Science, 38(1), 1–21. https://doi.org/10.1007/s11251-008-9075-4 Miguel-Revilla, D., & Sánchez-Agustí, M. (2018). Conciencia histórica y memoria colectiva: marcos de análisis para la educación histórica [Historical consciousness and collective memory: Analytical frameworks for historical education]. Revista De Estudios Sociales, 65, 113–125. https://doi.org/10.7440/res65.2018.10 Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054. https://doi.org/10.1111/j.1467- 9620.2006.00684.x Mishra, P., & Mehta, R. (2017). What we educators get wrong about 21st-century learning: Results of a survey. Journal of Digital Learning in Teacher Education, 33(1), 6–19. https://doi.org/10.1080/21532974.2016.1242392 Neuman, W. L. (2007). Basics of social research. Qualitative and quantitative approaches (2nd ed.). London, UK: Pearson. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). London, UK: McGraw-Hill. Redecker, C., & Punie, Y. (Eds.). (2017). European framework for the digital competence of educators: DigCompEdu. Luxembourg: Publications Office of the European Union. https://doi.org/10.2760/159770 Roig-Vila, R., Mengual-Andrés, S., & Quinto-Medrano, P. (2015). Conocimientos tecnológicos, pedagógicos y disciplinares del profesorado de primaria [Primary teachers’ technological, pedagogical and content knowledge]. Comunicar, 23(45), 151–159. https://doi.org/10.3916/C45-2015-16 Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technological pedagogical content knowledge (TPACK): The development and validation of an assessment instrument for preservice teachers. Journal of Research on Technology in Education, 42(2), 123–149. https://doi.org/10.1080/15391523.2009.10782544 Seixas, P. (2017). A model of historical thinking. Educational Philosophy and Theory, 49(6), 593–605. https://doi.org/10.1080/00131857.2015.1101363 Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin Company. Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. https://doi.org/10.3102/0013189X015002004 Valtonen, T., Kukkonen, J., Kontkanen, S., Mäkitalo-Siegl, K., & Sointu, E. (2018). Differences in pre- service teachers’ knowledge and readiness to use ICT in education. Journal of Computer Assisted Learning, 34(2), 174–182. https://doi.org/10.1111/jcal.12225 Valtonen, T., Sointu, E., Kukkonen, J., Kontkanen, S., Lambert, M. C., & Mäkitalo-Siegl, K. (2017). TPACK updated to measure pre-service teachers’ twenty-first century skills. Australasian Journal of Educational Technology, 33(3), 15–31. https://doi.org/10.14742/ajet.3518 van Boxtel, C., & van Drie, J. (2018). Historical reasoning: Conceptualizations and educational applications. In S. Alan Metzger & L. M. Harris (Eds.), The Wiley international handbook of history teaching and learning (pp. 149–176). New York, NY: Wiley-Blackwell. https://doi.org/10.14742/ajet.3907 https://doi.org/10.14742/ajet.282 https://www.jstor.org/stable/26196124 https://www.citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge/ https://www.citejournal.org/volume-9/issue-1-09/general/what-is-technological-pedagogicalcontent-knowledge/ https://www.j-ets.net/ETS/issues619c.html?id=62 https://doi.org/10.1007/s11251-008-9075-4 https://doi.org/10.7440/res65.2018.10 https://doi.org/10.1111/j.1467-9620.2006.00684.x https://doi.org/10.1111/j.1467-9620.2006.00684.x https://doi.org/10.1080/21532974.2016.1242392 https://doi.org/10.2760/159770 https://doi.org/10.3916/C45-2015-16 https://doi.org/10.1080/15391523.2009.10782544 https://doi.org/10.1080/00131857.2015.1101363 https://doi.org/10.3102/0013189X015002004 https://doi.org/10.1111/jcal.12225 https://doi.org/10.14742/ajet.3518 Australasian Journal of Educational Technology, 2020, 36(2). 12 Van Vaerenewyck, L. M., Shinas, V. H., & Steckel, B. (2017). Sarah’s story: One teacher’s enactment of TPACK+ in a history classroom. Literacy Research and Instruction, 56(2), 158–175. https://doi.org/10.1080/19388071.2016.1269267 Voogt, J., & Roblin, N. P. (2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299–321. https://doi.org/10.1080/00220272.2012.668938 Wellington, J. (2015). Educational research: Contemporary issues and practical approaches (2nd ed.). London, UK: Bloomsbury. Willermark, S. (2018). Technological pedagogical and content knowledge: A review of empirical studies published from 2011 to 2016. Journal of Educational Computing Research, 56(3), 315–343. https://doi.org/10.1177/0735633117713114 Zhao, Y., & Lei, J. (2009). New technology. In G. Sykes, B. Schneider, & D. N. Plank (Eds.), Handbook of education policy research (pp. 671–693). London, UK: Routledge. https://doi.org/10.4324/9780203880968 Corresponding author: Diego Miguel-Revilla, dmigrev@sdcs.uva.es Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: Miguel-Revilla, D., Martínez-Ferreira, J. M., & Sánchez-Agustí, M. (2020). Assessing the digital competence of educators in social studies: An analysis in initial teacher training using the TPACK-21 model. Australasian Journal of Educational Technology, 36(2), 1-12. https://doi.org/10.14742/ajet.5281 https://doi.org/10.1080/19388071.2016.1269267 https://doi.org/10.1080/00220272.2012.668938 https://doi.org/10.1177/0735633117713114 https://doi.org/10.4324/9780203880968 mailto:dmigrev@sdcs.uva.es https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.5281 Introduction and literature review Purpose of the study Methodological approach Context and participants Instrument, data processing, and analysis Design of the intervention Results An examination of the TPACK-21 model Starting point and progress after the intervention Discussion and conclusions Limitations and future directions Acknowledgments References