Australasian Journal of Educational Technology, 2021, 37(3). 152 Exploring preservice teacher self-reported and enacted TPACK after participating in a learning activity types short course Daniel James Mourlam, Steven Randall Chesnut University of South Dakota Heather Bleecker Salish Kootenai College Preparing preservice teachers to teach effectively with digital technologies and media (DTM) is at the heart of initial teacher preparation as digital access continues to increase throughout society. The development of preservice teacher technological pedagogical content knowledge (TPACK) is important in order to create DTM-rich instruction. This study investigated preservice teacher self-reported and enacted TPACK after participating in learning activity types short courses, and the relationship between preservice teacher perceptions and observed practice. Data collection occurred using a pre-post self-report survey and preservice teacher lesson plans collected before and after short course implementation in two educational technology courses. Results from structural equation modeling and confirmatory factor analysis indicated no significant differences in preservice teacher self-reported technological content knowledge (TCK), technological pedagogical knowledge (TPK), and TPACK, significant differences in preservice teacher enacted TCK, TPK, and TPACK, and no significant relationship between preservice teacher self-reported and enacted TPACK. Implications for practice or policy: • Teacher educators may consider using learning activity types short courses for preservice teachers’ enacted TPACK development, while using other strategies for self- reported TPACK development. • Teacher education programs can determine preservice teachers’ readiness for teaching with DTM by assessing both preservice teachers’ knowledge and beliefs. • Teacher education programs may consider using both self-report and performance-based assessments of preservice teachers’ knowledge as there may be discrepancies in what preservice teachers believe and how they enact their knowledge in practice. Keywords: technological pedagogical content knowledge, teacher education, digital technology and media, educational technology, structural equation modeling Introduction Preparing preservice teachers to effectively teach with digital technologies and media (DTM) has become an important priority in today’s society (Herring et al., 2015; US Department of Education, 2017). Central to this effort has been the focus on developing preservice teachers’ integrated knowledge of content, pedagogy, and technologies (Voogt et al., 2016). The purpose of this study was to explore the nature of preservice teacher technological pedagogical content knowledge (TPACK) development after participating in a Learning Activity Types Short Course. This study begins by exploring the existing literature on TPACK and preparing teachers to teach with technology, before describing the study and results. Technological pedagogical content knowledge Shulman’s (1986; 1987) professional knowledge base of teaching encompassed multiple areas, including content knowledge (CK), pedagogical knowledge (PK), curriculum knowledge, pedagogical content knowledge (PCK), knowledge of learners, knowledge of educational contexts, and knowledge of the purpose and value of education. With the emergence of new DTM and their target for integration into learning, of specific interest is Australasian Journal of Educational Technology, 2021, 37(3). 153 pedagogical content knowledge and its more recent reconceptualisation as TPACK, a framework for teacher knowledge needed to effectively teach with DTM (Angeli et al., 2016; Koehler & Mishra, 2008; Voogt et al., 2016). The TPACK framework is comprised of seven overlapping knowledge domains (Figure 1, Table 1) (Archambault, 2016; Koehler & Mishra, 2008; Mishra & Koehler, 2006). Figure 1. The TPACK Framework, reproduced by permission of the publisher, © 2012 by tpack.org Table 1 Definitions of TPACK domains TPACK domain Knowledge of: Technological knowledge (TK) Technologies and how they function Content knowledge (CK) Subject-matter to be learned Pedagogical knowledge (PK) Teaching and learning Pedagogical content knowledge (PCK) How to teach content using specific instructional strategies Technological content knowledge (TCK) How content topics can be represented in different ways using technologies Technological pedagogical knowledge (TPK) How technologies can be used to support teaching and learning activities Technological pedagogical content knowledge (TPACK) How to effectively teach content using specific strategies and technologies Mishra and Koehler (2006) explained that TPACK is context-bound where no DTM solution can be applied to every situation. Yet, teachers’ work spans several contexts including micro (classroom), meso (school), and macro (state and national) levels, each of which influence teachers’ knowledge and the development of that knowledge (Porras-Hernández & Salinas-Amescua, 2013; Rosenberg & Koehler, 2015). Given the complexity of context within TPACK and the lack of research published in this area (Rosenberg & Koehler, 2015), this continues to be an area for continued research. Conceptual iterations of TPACK The TPACK framework has been reconceptualised multiple times since the early 2000s (Angeli et al., 2016; Mishra, 2019; Saad et al., 2012; Yeh et al., 2014). As a result of these multiple conceptualisations, the theoretical conceptualisation of TPACK has become fuzzy (Phillips & Harris, 2018). In an effort to refine TPACK to become more comprehensive of teachers’ knowledge, there has been “an attempt to represent all of teachers’ knowledge, reasoning, and action” (Phillips & Harris, 2018, p. 2113), extending beyond Shulman’s (1986; 1987) original intent. It has been suggested that TPACK operates as a temporary framework that guides Australasian Journal of Educational Technology, 2021, 37(3). 154 teachers’ exploration and use of new and unfamiliar DTM during instruction, which may indicate TPACK fits within the broader knowledge base for teaching (Cox, 2008; Phillips & Harris, 2018). Therefore, PCK represents a teacher’s ability to use familiar DTM during instruction, whereas TPACK represents the “specific nature of a teacher’s PCK when unfamiliar tools are considered and implemented for educational purposes” (p. 2114). Contributing to the fuzziness of TPACK has been the difficulty measuring TPACK using self-report instruments where items created to measure one domain load onto a different domain during analyses (e.g., Archambault & Barnett, 2010; Burgoyne et al., 2010; Chai et al,, 2016). This may be due to the socially and systematically biased nature of self-report survey methodology (Specter, 1994). Therefore, it is recommended multiple measures be used to provide additional validity in TPACK studies (Abbitt, 2011; Cavanaugh & Koehler, 2013; Chai et al., 2016; Graham, 2011). This can be accomplished using ratings of teachers’ lesson planning, observed instruction, and interviews. The expected result is a relationship between multiple measures of TPACK, which has proven elusive in prior studies found in the literature (Hofer & Grandgenett, 2012; Kopcha et al., 2014; So & Kim, 2009). Currently, to the best of the authors’ knowledge, only one study has found a significant positive correlation between preservice teacher self-reported TPACK (TPACK survey) and their enacted TPACK through lesson planning (Deng et al., 2017). Clearly, as has been identified in the literature (Chai et al., 2016) this is an area needing additional research and was an area of focus in this study. Preparing preservice teachers for DTM integration As educators prepare to teach with DTM, they encounter both external and internal barriers and therefore have a variety of needs that extend beyond their knowledge and skills. Although the impact of external barriers to DTM integration (e.g., DTM access) have been reduced (Ertmer, 1999) as schools have invested in DTM in recent years, widespread integration still remains elusive and ineffective for many; a consistent trend in education (Cuban, 2001; Delgado et al., 2015; Ertmer et al., 2012; Koh et al., 2015). It is theorised DTM integration is driven by internal barriers, such as teacher beliefs and attitudes (Ertmer & Ottenbreit-Leftwich, 2010; Ertmer et al., 2012; Hew & Brush, 2007), which may be challenged by reconceptualisations of effective teaching with DTM that typically include learner-centered approaches (Tondeur, van Braak, & Ertmer, 2017). This may create a sense of dissonance when teachers learn and attempt DTM integration (Ertmer, 2005; Ertmer & Ottenbreit-Leftwich, 2010) ultimately influencing how they teach with DTM (Ertmer et al., 2015; Hsu, 2016; Overbay et al., 2010; Tondeur et al., 2017). Therefore, teacher education programs must focus on shaping preservice teachers’ DTM beliefs and attitudes in addition to developing the knowledge needed to create DTM- rich learning experiences. This can be achieved by engaging preservice teachers in mastery learning experiences where they have opportunities to make appraisals of their own skills and the value of authentic teaching practices (Bandura, 1986, 2001), such as planning and implementing DTM-rich lessons. In preservice teacher education programs, standalone educational technology courses have often been used to develop preservice teacher TPACK. In these courses, preservice teachers often create DTM-rich lessons (An et al., 2011; Angeli & Valanides, 2013; Figg & Jaipal, 2013; Hofer & Harris, 2010; Koehler et al., 2011; Mouza, 2016; Mouza et al., 2017), where they either: (a) explore the pedagogical affordances of DTM before examining how they could be combined with content through lesson development (Angeli & Valanides, 2013); (b) develop both their PCK and TPACK concurrently during lesson development (Brush & Saye, 2009; Koehler et al., 2011; Mishra & Koehler, 2006); or (c) develop sound instructional objectives and strategies, before subsequently examining how DTM can support those objectives (Harris & Hofer, 2009; Harris et al., 2009; Hofer & Harris, 2016; Niess et al., 2010). The primary differences among these three approaches is the point at which DTM is considered in the instructional design process. The latter of which waits to consider the role of DTM in the lesson until decisions are made regarding content goals and instructional strategies. In doing so, over emphasis on DTM can be avoided, allowing for TPACK to be developed authentically as part of the planning process aligned with curricular standards (Harris & Hofer, 2009). Therefore, this latter approach to TPACK development was the focus of this study, using the learning activity types (LATs) (Harris & Hofer, 2009; Hofer & Harris, 2010). Australasian Journal of Educational Technology, 2021, 37(3). 155 LATs are taxonomies of content specific instructional activities with aligned DTM, which assist teachers with integrating DTM during lesson planning (Harris & Hofer, 2009; Hofer & Harris, 2010). According to Hofer and Harris (2010), preservice teachers: (a) plan content goals; (b) select and sequence multiple LATs and assessment strategies to engage students in learning; and (c) consider the role DTM might have in the lesson. This develops preservice teacher TPACK authentically, by aligning DTM use with content and pedagogical needs (Hofer & Harris, 2010). Few studies have explored LATs use in preservice teacher education and inservice teacher professional development. Karns (2019) investigated the use of the LATs short course as part of a collaborative curriculum design-based professional development experience for teachers and found that teachers’ technological pedagogical knowledge (TPK), technological content knowledge (TCK), and TPACK changed in ways that allowed them to enhance their instruction through DTM. Figg and Jaipal’s (2013) study on the TPACK-in- Practice workshop, used LATs during the initial stage of the 4-stage workshop. In this first stage, LATs were modeled for preservice teachers so that “they are able to visualise how learners use the tools … as learners to achieve a learning goal” (p. 5041-5042). Preservice teachers then engaged in pedagogic dialogue, discussing how DTM was integrated into instruction, allowing them to understand how “pedagogy, content, and technology work together in the instruction” (p. 5042). Preservice teachers then learned more about the DTM, before creating and teaching a similar learning experience at a K-8 teacher workshop. Findings from preservice teachers’ comments indicated they believed they were more confident in promoting DTM-rich instruction and considered how DTM could be used across curricular areas. To assist teachers, Hofer and Harris (2016) created LATs short courses consisting of eight brief, developmental, video-based modules. In LATs short courses, preservice teachers: (a) think about how they have used DTM during learning experiences; (b) analyse lesson plans for instructional goals, and technologies; (c) explore LATs to determine DTM and activity substitutions including how they would change lesson experiences; and (d) create their own lesson plan using the LATs taxonomies. As little research has been published regarding the LATs, the short courses, and their efficacy for developing preservice teacher knowledge, these are investigated in this study. Purpose of the study The purpose of this study was to examine the efficacy of the LATs short course in developing preservice teacher self-reported and enacted TCK, TPK, and TPACK. Given the focus of the LATs short courses on supporting preservice teacher technology integration, only TPACK domains where (technological knowledge) TK intersects with either PK or CK were included within the scope of this study. A repeated measures design was employed to answer the following research questions: 1. To what extent does self-reported TPACK change after an LATs short course? 2. To what extent does enacted TPACK change after an LATs short course? 3. To what extent do self-report and enacted TPACK correlate before and after an LATs short course? Methods Study participants were undergraduates completing one of two educational technology (ET) courses at a medium-sized research university in the Midwest United States of America. The first course, ET1, was an introductory educational technology course required for admission to the teacher education program. ET1 introduces digital citizenship, social media, distance education, and digital storytelling, in addition to DTM that supports teaching and learning in elementary and secondary classrooms. The second course, ET2, was an educational technology methods course completed during the first semester of a year-long teacher residency program. ET2 develops preservice teachers’ DTM skills and confidence for integrating DTM in their instruction, focused on the 4Cs (i.e., collaboration, communication, critical thinking, and creativity), classroom management, troubleshooting, and reflecting on DTM-rich instruction. In each course, LATs short course integration was completed by one of the authors. Australasian Journal of Educational Technology, 2021, 37(3). 156 In the combined courses, 80 preservice teachers participated, with slightly more than half recruited from ET2 (N = 46; 57.5%). The sample was representative of the population in the teacher education program Table 2 shows the demographic information. Most preservice teachers were elementary education majors (N = 72, 90.0%), with fewer in secondary education (N = 6, 7.5%). Many elementary education majors were also double majoring with special education (N = 27, 33.8%). Additionally, all preservice teachers enrolled in ET2 previously completed the ET1 course. Table 2 Participant information disaggregated by educational technology course Participant demographics Total ET1 ET2 Gender Male 12 (15.0%) 9 (26.5%) 3 (6.5%) Female 68 (85.0%) 25 (73.5%) 43 (93.5%) Age Younger than 22 years 66 (83.5%) 30 (90.9%) 36 (78.3%) Between 23 and 26 years 11 (13.9%) 2 (6.1%) 9 (19.6%) Between 27 and 32 years 1 (1.3%) 0 (0.0%) 1 (2.2%) Older than 33 years 1 (1.3%) 1 (3.0%) 0 (0.0%) Academic classification Freshman 2 (2.5%) 2 (5.9%) 0 (0.0%) Sophomore 26 (32.5%) 26 (76.5%) 0 (0.0%) Junior 6 (7.5%) 6 (17.6%) 0 (0.0%) Senior 46 (57.5%) 0 (0.0%) 46 (100%) Instructional context In both ET1 and ET2, LATs short courses were implemented following the recommendations of Hofer and Harris (2016) where preservice teachers considered how they had previously used DTM during learning experiences, before analysing lesson plans for instructional goals and technologies, and exploring LATs to determine DTM selection and activity substitutions. In both ET1 and ET2, LATs short course implementation culminated with preservice teachers creating their own lesson plans using the LATs taxonomies. During lesson planning, preservice teachers began by first developing instructional goals and strategies, before then considering the use of DTM in the lesson (Harris & Hofer, 2009; Harris et al., 2009; Hofer & Harris, 2016; Niess et al., 2010). Given the unique needs of preservice teachers enrolled in ET1 and ET2 in relation to their progression in the teacher education program, LATs short course implementation was further customised to meet the needs of each context and is described in detail in the following paragraphs. In ET1, the LATs short course was embedded within a unit on distance education (Figure 2). In this unit, preservice teachers learned about multiple approaches to blended and online teaching, global learning, as well as DTM that can facilitate distance education in K-12 classrooms. The LATs short course spanned 3 weeks and was taught in a hybrid format with one face-to-face class meeting at the beginning and end of the unit, with all remaining instruction occurring asynchronously online through a course management system. Each week preservice teachers participated in course activities for approximately 4 hours. In the first week, students were introduced to distance education in K-12 schools, completed readings in blended and online learning, and further explored the hybrid structure of the distance education unit they were beginning. Throughout that week students participated in the LAT short course, asynchronous online discussions, small group and individual projects, and analysed lesson plans to identify relationships between learning goals, activities, and DTM. In the second week, students completed readings in global learning and completed LATs short course modules. Throughout that week students participated in activities and asynchronous online discussions where they actively deconstructed the LATs taxonomies, comparing them to Bloom’s taxonomy, and exploring Australasian Journal of Educational Technology, 2021, 37(3). 157 opportunities to leverage DTM to modify lesson plans to support higher order thinking and one of the 4Cs. In the final week, students completed the remainder of the short course and created a lesson plan that utilised a blended, online, or global learning approach. Throughout that week students further discussed distance education, their developed lesson plans, and brainstormed the integration of distance education in their future classrooms. Figure 2. LATs short course implementation in ET1 In ET2, the LATs short course followed a similar, although slightly different implementation, with preservice teachers completing the short course over 4 weeks as part of a unit on technology integration (Figure 3). Each week, preservice teachers spent approximately 2 hours participating in course activities. During the first 2 weeks, taught online, preservice teachers completed LATs short course modules and participated in asynchronous online discussions with their peers. During the discussion, preservice teachers selected a lesson taught by their mentor teacher and described the learning objective, content focus, instructional strategies used, as well as the roles of both students and the teacher. Preservice teachers also described the role of DTM in the lesson, if any. Preservice teachers then, as a small group, selected one lesson to enhance using DTM. As part of this discussion preservice teachers modified lessons such that students would be engaged in one or more of the 4Cs. At the end of the first 2 weeks, preservice teachers completed the remainder of the short course modules and began preparing to plan, teach, and evaluate a DTM-rich lesson. During the final 2 weeks, preservice teachers completed the remaining short course modules and engaged in an asynchronous online discussion about teaching with technologies. Preservice teachers posted one-to-two issues, concerns, or questions they had and then worked as a small group to find solutions to each group member’s concerns. Preservice teachers had approximately 5 weeks to plan, teach, and evaluate their DTM-rich lesson at their year-long teacher residency placement. Figure 3. LATs short course implementation in ET2 Procedure and data collection As part of accreditation and tracking efforts in the teacher education program, extensive data were collected throughout both courses. Approval to use the data was granted by the university’s institutional review board (IRB). Data on student progress at the intersection of technology, pedagogy, and content were collected pre- LATs and once again post-LATs. A TPACK survey instrument (Pamuk et al., 2013) was used to measure self- reported knowledge and was administered at the beginning of the short course and then again after completing Australasian Journal of Educational Technology, 2021, 37(3). 158 the short course and lesson plan assignment. The instrument was comprised of 37 4-point Likert-type questions that align with the seven TPACK domains. Participants responded to prompts on the TPACK instrument with ratings from (1) strongly disagree to (4) strongly agree. The internal consistency of the items used in this study was similar to those of Pamuk et al. (2013), with Cronbach’s alphas for the TPACK domains ranging from .83 to .90 for the pre-LATs measure and .89 to .96 for the post-LATs measure. This instrument underwent both content and construct validity testing, where Pamuk et al. investigated the interactions among the TPACK knowledge bases. Results from structural equation modelling found that TK, CK, and PK had almost no direct effects on TPACK development, while PCK’s effect was much lower than TPK and TCK. Given the goal of this study to focus on TCK, TPK, and TPACK domains, this instrument was best suited. Additionally, the lesson plans created by preservice teachers before and after the LATs short course were analysed for enacted TPACK using the Technology Integration Assessment Rubric (Harris et al., 2010). The rubric was designed to assess lesson plans for teachers’ enacted knowledge in TCK, TPK, and two dimensions of TPACK: (1) technology selections or the compatibility of technologies with the curricular goals and instructional strategies identified in a lesson plan; and (2) fit or how strongly connected the content, pedagogy, and technology are within the lesson plan. For each knowledge domain, the rubric included four levels of performance (e.g., TPK ranging from 1 = technology use does not support instructional strategies to 4 = technology use optimally supports instructional strategies). The rubric aligned well with both the goals of the study and the self-report TPACK survey instrument. Additionally, the rubric underwent both content validity and reliability testing. In this study, two raters are experienced TPACK researchers with extensive knowledge of the TPACK framework and experience teaching with DTM. The raters first calibrated their scoring by evaluating three lesson plans from another educational technology course, wherein the researchers independently scored each lesson plan and discussed their ratings prior to moving on to the next. To assist with independent evaluation of the preservice teachers’ lesson plans, the researchers collaboratively selected exemplars for each rubric criterion. The researchers independently scored the remainder of the lesson plans. Even with variability in the focus and scope of the lesson plans submitted by students and the four levels of performance, the Cohen’s Kappa suggested that the raters demonstrated slight to fair amount of agreement across the areas of curriculum goals (K = 0.08), instructional strategies (K = 0.17), technology selection (K = 0.18) and fit (K = 0.27) (see also Landis and Koch [1977] on levels of inter-rater agreement). Most of the disagreement was a single level difference in performance. For example, when rating, “Students will read a story on their tablets and then create a flip book stating who, what, where, when, why and how the story happened.”, one reviewer rated a 3 and the other a 4. To improve the accuracy of ratings, the researchers met after independently scoring the lesson plans to compare ratings, consolidate evidence for the ratings, and negotiate differences to ensure complete agreement in enacted TPACK (see also Creswell [2008] for procedures pertaining to consolidation in the coding process). Data analysis Before the data were analysed, only participants with self-report and enacted TPACK data for both time points were retained (N = 78). Given the strong internal consistency of the self-report TPACK measure and the low rates of missing data from pre- and post-surveys (.02%), responses were imputed using a combination of commonly used imputation procedures using the mice package (van Buuren & Groothuis-Oudshoorn, 2011) in R (R Core Team, 2019). To answer the research questions about the development of TCK, TPK, and TPACK throughout the LATs short course, longitudinal structural equation models (SEM) were analysed in the lavaan package (Rosseel, 2012) to model the latent variables measured by the TPACK self-report alongside the expert ratings of enacted TPACK. For each of the three intersecting domains (i.e., TCK, TPK, & TPACK), a separate confirmatory factor analysis and SEM model were specified using the original items from the corresponding domains of the TPACK self- report measure, except for the overarching TPACK domain. For the TPACK domain, three parcels were created using the serpentine approach based on the results of the factor loadings from the confirmatory factor analysis to reduce item error and enhance model fit (Little, 2013; Little et al., 2002). Additionally, in each of the SEM models, the corresponding enacted TPACK ratings in a latent variable framework was included to not only model the longitudinal data, but as correlates of the TPACK self-report latent variables. Traditionally, a paired- Australasian Journal of Educational Technology, 2021, 37(3). 159 samples t-test would suffice for examining the extent of mean difference between a pre- and post- condition. However, to ensure the complexity of the models were captured, this study leveraged one of the features of strong factorial invariance to test mean difference from pre-LAT to post-LAT. More specifically, in the final step of longitudinal factorial invariance, item intercepts are constrained over time and the latent variable means are freed after the first wave. The constraining of the first latent mean to 0 and free estimation of latent means after, along with constrained item intercepts across all waves, forces any differences in response patterns in subsequent waves to the latent variable, thus providing a mean difference from the first wave. The difference between this approach to mean differences and a traditional t-test is that the latent model removes the measurement error, providing a more accurate insight into the true differences of the construct. Structural paths were interpreted according to the following effect size criteria: greater than .05, small; greater than .10, moderate; and greater than .25, large (Keith, 2019). Figure 4 illustrates the structure and constraints for the TCK panel model, which is similar to the TPK and TPACK panel models. Figure 4. TCK SEM model To evaluate the model fit of the confirmatory factor analysis and the SEM models, the χ2 estimate, comparative fit index (CFI), root mean square error of association (RMSEA), and standardised root mean square residual (SRMR) were utilised. Acceptable criteria for model fit were a χ2 estimate that was not statistically significant, a CFI ≥ .95, an RMSEA ≤ .06, and an SRMR ≤ .08 (Hu & Bentler, 1999; Schermelleh-Engel et al., 2003). Given that the data were longitudinally modeled, longitudinal factorial invariance was examined to ensure that the latent variables represented the same constructs across the two time points. Factorial invariance is satisfied if the Δχ2 is not statistically significant and/or when the ΔCFI ≤ .01 between each step (Cheung & Rensvold, 2002). The models successfully passed from configural (no constraints), to weak (loadings), and strong (item intercepts) levels of longitudinal factorial invariance. Figure 4 illustrates these constraints on the loadings with λ and the intercepts with τ. Results Before reporting the results of the SEM analyses, this study first addressed concerns that were likely to emerge by including both ET1 and ET2 samples in the same analytic models. First, by including participants from both courses there might be systematic differences in responses to the self-reported and observed TPACK domains. Australasian Journal of Educational Technology, 2021, 37(3). 160 More specifically, one might assume that self-reported and observed scores might be higher in the ET2 group than in the ET1 group, as a function of the ET2 having already completed the ET1 course. To examine this possibility, the SEM models were modified to include the participants course as a predictor of both the self- report and enacted TPACK domains before and after the LAT short course. This establishes parameters that would mimic a t-test on latent variables, such that the regression coefficients would indicate average differences in those latent variables. Results from these models showed that a significant difference between ET1 and ET2 only emerged in the TPK model. Being enrolled in ET2 predicted significant differences in self-report TPK prior to the LAT short course, with those enrolled in ET2 reporting significantly higher TPK. Second, by including participants from both courses, there might be systematic differences in the strength and directionality of the relationships between self-reported and observed TPACK domains before and after the LAT short course. To examine this possibility, the SEM models were modified to focus on moderation by establishing them as multi-group SEM models, in which longitudinal factorial invariance (see also Little, 2013) was established prior to constraining parameter estimates between groups. In this type of analysis, improvements in model fit moving from a single-group focus to a multi-group focus (as indicated by changes in χ2) indicates the possibility of differences in factor structures or parameter estimates between groups. Additionally, drops in model fit (as indicated by changes in χ2) after constraining a parameter (e.g., regression from before to after LAT) between the two groups indicated differences in strength and directionality of that relationship. Results demonstrated that modeling the two groups separately resulted in significantly worse model fit than when modeled as a single group. Additionally, once modeled separately, constraints to the factor structure and regression and correlation parameters did not significantly alter model fit. These results indicated that there were no systematic differences in: (a) how the individual items make up the latent variables between both groups; (b) the relationships between enacted and observed TPACK domains before and after LAT between both groups; and (c) the regressions from before to after LAT latent variables between both groups. It is important to note the second modification lacked the sample size needed to be completely confident in the stability of estimates in a multigroup-SEM analysis (see also Little, 2013). As such, interpretations were exploratory and used to corroborate findings from the first model changes. Ultimately, the results from these two adjustments to the SEM analyses proposed to answer the research questions highlighted three points. First, out of the 12 possible times to occur, there was only 1 mean difference in the self-reported or observed latent variables for the TPACK domains before and after the LAT short course between ET1 and ET2 students. This was controlled for in the following analyses. Second, there were no differences in how the items used to measure self-reported or enacted TPACK domains correspond to their latent variable in both item-level intercepts and factor loadings between the two groups. Finally, there was no significant difference in the relationships between the self-reported and observed TPACK domains before and after the LAT short course that could suggest moderation due to course enrollment. Combined, these results indicated that it was statistically sound to model ET1 and ET2 together in the same SEM analyses. Longitudinal SEM The final SEM model for the TCK domain demonstrated acceptable model fit with χ2(35) = 43.73, p = .15, CFI = .98, RMSEA = .06, SRMR = .06 (Table 3). Responses to the post-LATs measure were strongly predicted by responses to the pre-LATs measure (β = .446). There was no statistically significant difference in self-report TCK between pre- and post-LATs measurements, with a raw score mean difference of 0.050. Concerning the rater TCK, ratings to students’ lesson plans post-LAT were not significantly predicted by ratings to students’ lesson plans pre-LATs (β = -.019). However, results from the constraints suggested that ratings of student lesson plans significantly improved post-LATs, with a raw score mean difference of 0.321. When considering the self- report and the raters measures of TCK, results from the panel model suggested that there was no significant relationship between self-report and raters pre-LATs (r = -.087) or post-LATs (r = .003). Australasian Journal of Educational Technology, 2021, 37(3). 161 Table 3 Model parameters for the TCK domain Parameter B SE p – value β Loadings SR TCK Time 1 by Pre.TCK1 (𝜆1, 𝜏1 ) .441 .046 < .001 .766 Pre.TCK2 (𝜆2, 𝜏2 ) .427 .044 < .001 .834 Pre.TCK3 (𝜆3, 𝜏3 ) .375 .044 < .001 .672 Pre.TCK4 (𝜆4, 𝜏4 ) .437 .046 < .001 .756 SR TCK Time 2 by Post.TCK1 (𝜆1, 𝜏1) .441 .046 < .001 .919 Post.TCK2 (𝜆2, 𝜏2) .427 .044 < .001 .839 Post.TCK3 (𝜆3, 𝜏3) .375 .044 < .001 .816 Post.TCK4 (𝜆4, 𝜏4) .437 .046 < .001 .921 EN TCK Time 1 by Pre.TCK Rating (𝜏5) 1.000 1.000 EN TCK Time 2 by Post.TCK Rating (𝜏5) 1.000 1.000 Regressions SR TCK Time 2 on SR TCK Time 1 .553 .144 < .001 .446 EN TCK Time 2 on EN TCK Time 1 - .019 .086 .823 (NS) - .025 Covariances SR TCK Time 1 with EN TCK Time 1 - .076 .107 .476 (NS) - .087 SR TCK Time 2 with EN TCK Time 2 .003 .088 .972 (NS) .004 Latent variable means SR TCK Time 1 constrained to 0 .000 .000 SR TCK Time 2 freely estimated - .050 .148 .737 (NS) - .040 EN TCK Time 1 constrained to 0 .000 .000 EN TCK Time 2 freely estimated .321 .126 < .050 .482 Note. SR = self-report; EN = enacted; NS = not statistically significant The final SEM model for the TPK domain demonstrated acceptable model fit with χ2(34) = 31.07, p = .61, CFI = 1.00, RMSEA = .00, SRMR = .06 (Table 4). Responses to the post-LATs measure were significantly predicted by responses to the pre-LATs measure (β = .399). There was no statistically significant difference in self-report TPK between the pre- and post-LATs measurements, with a raw score mean difference of .044. Concerning the rater TPK, ratings of students’ lesson plans post-LATs were not significantly predicted by ratings of students’ lesson plans pre-LATs (β = .034). However, results from the constraints suggested that ratings of student lesson plans significantly improved post-LATs, with a raw score mean difference of 0.513 (p < .001). When considering the self-report and the raters measures of TPK, results from the panel model suggested that there was no significant relationship between self-report and raters pre-LATs (r = -.003) or post-LATs (r = -.108). Australasian Journal of Educational Technology, 2021, 37(3). 162 Table 4 Model parameters for the TPK domain Parameter b SE p – value β Loadings: Constraints illustrated in specified model SR TPK Time 1 by Pre.TPK1 (𝜆1, 𝜏𝑏) .358 .040 < .001 .672 Pre.TPK2 (𝜆2, 𝜏𝑐) .417 .044 < .001 .748 Pre.TPK3 (𝜆3, 𝜏𝑑) .404 .045 < .001 .726 Pre.TPK4 (𝜆4, 𝜏𝑒) .452 .045 < .001 .842 SR TPK Time 2 by Post.TPK1 (𝜆1, 𝜏𝑏 ) .358 .040 < .001 .771 Post.TPK2 (𝜆2, 𝜏𝑐 ) .417 .044 < .001 .862 Post.TPK3 (𝜆3, 𝜏𝑑 ) .404 .045 < .001 .857 Post.TPK4 (𝜆4, 𝜏𝑒 ) .452 .045 < .001 .948 EN TPK Time 1 by Pre.TPK Rating (𝜏𝑎 ) 1.000 1.000 EN TPK Time 2 by Post.TPK Rating (𝜏𝑏 ) 1.000 1.000 Regressions SR TPK Time 2 on SR TPK Time 1 .497 0.149 < .001 .399 EN TPK Time 2 on EN TPK Time 1 .022 .075 .766 (NS) .034 Covariances SR TPK Time 1 with EN TPK Time 1 - .003 .112 .981 (NS) - .003 SR TPK Time 2 with EN TPK Time 2 - .076 .085 .371 (NS) - .108 Latent variable means SR TPK Time 1 constrained to 0 .000 .000 SR TPK Time 2 freely estimated .044 .155 .775 (NS) .036 EN TPK Time 1 constrained to 0 .000 .000 EN TPK Time 2 freely estimated .513 .123 < .001 .834 Note. SR = self-report; EN = enacted; NS = not statistically significant The final SEM model for the TPACK domain demonstrated acceptable model fit with χ2(34) = 26.94, p = .80, CFI = 1.00, RMSEA = .00, SRMR = .08 (Table 5). Responses to the post-LATs measure were significantly predicted by responses to the pre-LATs measure (β = .492). There was not a statistically significant difference in self-report TPACK between the pre- and post-LATs measurement, with a raw score mean difference of – 0.012. Concerning the rater TPACK, ratings to students’ lesson plans post-LATs were not significantly predicted by ratings of students’ lesson plans pre-LATs (β = -.002). However, results from the constraints suggested that ratings of student lesson plans significantly improved post-LATs, with a raw score mean difference of 0.564. When considering the self-report and the raters measures of TPACK, results from the panel model suggested that there was not a significant relationship between self-report and raters pre-LATs (r = .046) or post-LATs (r = .083). Australasian Journal of Educational Technology, 2021, 37(3). 163 Table 5 Model parameters for the TPACK domain Parameter b SE p – value β Loadings: Constraints illustrated in specified model SR TPACK Time 1 by Pre.TPACK1 (parcel: 𝜆3, 𝜏𝑐 ) .384 .036 < .001 .822 Pre.TPACK2 (parcel: 𝜆4, 𝜏𝑑 ) .410 .038 < .001 .870 Pre.TPACK3 (parcel: 𝜆5, 𝜏𝑒 ) .368 .035 < .001 .881 SR TPACK Time 2 by Post.TPACK1 (parcel: 𝜆3, 𝜏𝑐) .384 .036 < .001 .949 Post.TPACK2 (parcel: 𝜆4, 𝜏𝑑) .410 .038 < .001 .946 Post.TPACK3 (parcel: 𝜆5, 𝜏𝑒) .368 .035 < .001 .932 EN TPACK Time 1 by Pre.TPACK Rating 1 ( 𝜆1, 𝜏𝑎 ) .527 .100 < .001 .657 Pre.TPACK Rating 2 ( 𝜆2, 𝜏𝑏) .975 .184 < .001 1.000 EN TPACK Time 2 by Post.TPACK Rating 1 ( 𝜆1, 𝜏𝑎 ) .527 .100 < .001 .509 Post TPACK Rating 2 ( 𝜆2, 𝜏𝑏 ) .975 .184 < .001 .993 Regressions SR TPACK Time 2 on SR TPACK Time 1 .688 .152 < .001 .492 EN TPACK Time 2 on EN TPACK Time 1 - .002 .075 .983 (NS) - .002 Covariances SR TPACK Time 1 with EN TPACK Time 1 .046 .112 .679 (NS) .046 SR TPACK Time 2 with EN TPACK Time 2 .072 .103 .488 (NS) .083 Latent variable means SR TPACK Time 1 constrained to 0 .000 .000 SR TPACK Time 2 freely estimated - .012 .150 .939 (NS) - .008 EN TPACK Time 1 constrained to 0 .000 .000 EN TPACK Time 2 freely estimated .564 .175 < .001 .800 Note. SR = self-report; EN = enacted; NS = not statistically significant Discussion This study had three primary goals: (1) to determine if there were any statistically significant differences in preservice teachers’ self-reported TPACK before and after a LATs short course; (2) to determine if there were any statistically significant differences in preservice teachers’ enacted TPACK before and after a LATs short course; and (3) to determine the relationship between preservice teachers’ self-reported TPACK and their enacted TPACK. The outcomes of these endeavours will be explored in the following sections, beginning with understanding preservice teacher knowledge. Understanding preservice teacher self-reported TPACK Regarding the first research question, analysis of preservice teachers’ self-reported TCK, TPK, and TPACK before and after LATs short course implementation indicated no statistically significant differences for TCK, TPK, or TPACK. This is an important finding, because after completing the LATs short course, the candidates in this study did not believe their knowledge had changed in these areas and represented an area for continued research and development. Several studies have demonstrated the importance of teacher beliefs as it relates to teaching effectively with DTM (Ertmer, 2005; Ertmer & Ottenbreit-Leftwich, 2010; Ertmer et al., 2012; Hew Australasian Journal of Educational Technology, 2021, 37(3). 164 & Brush, 2007; Tondeur et al., 2017). Therefore, a contribution to the field emerging from this study is that specific effort may need to be taken to address preservice teacher beliefs that extends beyond the scope of the LATs short course as implemented in this study. It is recommended that in future studies, researchers explore the potential of clinical field experiences where preservice teachers implement the DTM-rich lessons they create as part of the LATs short course. Doing so may provide additional mastery experiences for preservice teachers and help them better understand their knowledge and abilities (Bandura, 1986, 2001). Preservice teacher enacted TPACK changed The second research question examined any differences in preservice teacher enacted TCK, TPK, and TPACK. Analysis of rater assessments of preservice teacher enacted TPACK indicated statistically significant differences for TCK, TPK, and TPACK. Therefore, it appears even though preservice teachers did not believe their TPACK changed, their ability to implement their knowledge through lesson planning did change as a result of the LATs short course implementation in ET1 and ET2. These findings are important because it demonstrates the potential of the LATs short course as a TPACK development option for teacher educators, adding to and supporting the limited research that has been published in this area (Karns, 2019). However, more research and development are needed to fully understand how the LATs short course may or may not impact preservice teacher knowledge, as this study provided only a snapshot of one implementation technique with a unique sample not representative of all contexts and settings. Relationships between self-reported and enacted TPACK This study also explored the relationship between self-reported and enacted TPACK. Findings from this study indicated no statistically significant relationships regarding self-reported and enacted TCK, TPK, and TPACK, which was similar to results reported in studies found in the literature (Hofer & Grandgenett, 2012; So & Kim, 2009). One explanation for this finding could reside with a preservice teacher’s familiarity with the DTM integrated into their lessons, which may mean the DTM has been become integrated within their PCK and they no longer rely on their TPACK (Phillips & Harris, 2018). While this study did not control for such a variable, it is plausible if an unfamiliar DTM was used in the first lesson and then a more familiar DTM was used in the second lesson plan, increased enacted TPACK makes sense. Therefore, in future research, the prior experiences of preservice teachers with lesson DTM should be examined to determine the extent DTM familiarity is a factor in preservice teacher knowledge growth. The lack of relationship between self-report and enacted TPACK was interesting, because TPACK is often presented as a single theoretical construct, when in fact, it appears it may be more complex (Angeli & Valanides, 2009; Koehler & Mishra, 2008; Saad et al., 2012; Yeh et al., 2014). If the TPACK framework was a single unified model, it would be reasonable to assume when preservice teachers’ self-report their knowledge increase there would also be increases in their enacted knowledge. However, findings from this study indicated this relationship did not exist. As a result, TPACK remains a fuzzy conceptual framework (Phillips & Harris, 2018) likely more complex than previously discussed in the literature. This finding has important implications for teacher education programs, because preservice teachers need to be prepared to teach effectively with DTM upon entering the profession. TPACK represents one metric of a preservice teacher’s ability to teach effectively with DTM. Yet, if there are discrepancies between different measures of the same construct, then it is possible teacher education programs may not fully understand preservice teacher knowledge and abilities. Therefore, it is recommended teacher education programs seek more comprehensive measures of preservice teacher knowledge, inclusive of both self-report and enacted TPACK, and also extending beyond TPACK, to self-efficacy and teacher beliefs about the value of teaching with DTM. In addition to qualitative methods, this may generate previously unexplored concepts and factors. These measures may provide additional information about preservice teachers to be used to modify courses and experiences to further promote preservice teacher growth. Australasian Journal of Educational Technology, 2021, 37(3). 165 Recommendations for future research and limitations Several implications for future research emerged from this study. First, study findings appear to suggest TPACK is a multi-faceted construct, where what preservice teachers believe about their knowledge may not always align with how they apply their knowledge in practice, confirming similar findings already in the literature (Hofer & Grandgenett, 2012; So & Kim, 2009), while simultaneously being contradictory to others (Deng et al., 2017). This has important implications for future research, because as new approaches for developing preservice teacher knowledge to create DTM-rich experiences emerge, researchers need to look beyond self- reported assessments of preservice teacher knowledge, which is a methodology criticised for being socially and systematically biased (Specter, 1994). Not that self-reports of preservice teacher knowledge have no role, but it appears that conceptually, self-reported and enacted TPACK are different constructs and need further empirical examination to develop a better theoretical understanding of TPACK’s seemingly multiple layers. Additionally, given the uncertainty within the literature regarding the nature of the TPACK framework as a conceptual model of teacher knowledge, this study supports Angeli et al.’s (2016) idea that rather than developing new iterations of the TPACK framework, researchers seek to identify how existing frameworks emerge at different levels both in and out of the teacher’s practice. In doing so, it is possible to reduce the effort to map TPACK as a collection of all teacher knowledge (Phillips & Harris, 2018) and instead, provide structure to how TPACK can operate as a framework for understanding teacher knowledge as it emerges at multiple levels within teachers and within their teaching. A limitation of this study was the intervention differences due to course delivery methods that could not be accounted for during data analysis. One aspect of the study was the use of relatively smaller sample sizes, due to limitations with enrollment in the current programs. Results from the multigroup SEM indicated that those from ET1 and ET2 did not differ significantly in their responses to the measures or in how these measures were associated. Therefore, it is recommended that researchers conduct future studies with larger sample sizes to further examine this invariance between groups to avoid this issue. Doing so would provide a much clearer understanding of both preservice teacher self-report and enacted TPACK for those at different levels in the teacher education program. Finally, the manner in which this study modeled the enacted TPACK constructs (i.e., TCK, TPACK) with single indicators for the pre-LATs and post-LATs lessons plans requires further explanation. In forming a latent variable, it is common to use three measured indicators from which the common variance is combined to create an underlying construct. Because the latent variable is defined as the shared variance between the indicators, indicator-specific error is removed from the latent variable. Using a single indicator means that all of the information from the indicator is passed to the latent variable. This information not only includes important information about the TPACK domain, but also potential rater bias. Future research that aims to use latent variables to represent enacted TPACK would benefit from three independent, yet statistically reliable raters. The latent variables would then be defined as the shared variance between the raters, removing independent rater bias. References Abbitt, J. (2011). Measuring technological pedagogical content knowledge in preservice teacher education: A review of current methods and instruments. Journal of Research on Technology in Education, 43(4), 281– 300. https://doi.org/10.1080/15391523.2011.10782573 An, H., Wilder, H., & Lim, K. (2011). Preparing elementary pre-service teachers from a non-traditional student population to teach with technology. Computers in the Schools, 28(2), 170–193. https://doi.org/10.1080/07380569.2011.577888 Angeli, C., & Valanides, N. (2009). Epistemological and methodological issues for the conceptualization, development, and assessment of ICT-TPCK: Advances in technological pedagogical content knowledge (TPCK). Computers & Education, 52(1), 154–168. https://doi.org/10.1016/j.compedu.2008.07.006 https://doi.org/10.1080/15391523.2011.10782573 https://doi.org/10.1080/07380569.2011.577888 https://doi.org/10.1016/j.compedu.2008.07.006 Australasian Journal of Educational Technology, 2021, 37(3). 166 Angeli, C., & Valanides, N. (2013). Technology mapping: An approach for developing technological pedagogical content knowledge. Journal of Educational Computing Research, 48(2), 199–221. https://doi.org/10.2190/EC.48.2.e Angeli, C., Valanides, N., & Christodoulou, A. (2016). Theoretical considerations of technological pedagogical content knowledge. In M. C. Herring, M. J. Koehler, & P. Mishra (Eds.), Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 11-32). Routledge. Archambault, L. M. (2016). Exploring the use of qualitative methods to examine TPACK. In M. C. Herring, M. J. Koehler, & P. I. Mishra (Eds.), Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 65-86). Routledge. Archambault, L. M., & Barnett, J. H. (2010). Revisiting technological pedagogical content knowledge: Exploring the TPACK framework. Computers & Education, 55(4), 1656–1662. https://doi.org/10.1016/j.compedu.2010.07.009 Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice Hall. Bandura, A. (2001). Guide for constructing self-efficacy scales. Unpublished manuscript. Brush, T., & Saye, J. W. (2009). Strategies for preparing preservice social studies teachers to integrate technology effectively: Models and practices. Contemporary Issues in Technology and Teacher Education, 9(1), 46–59. https://citejournal.org/volume-9/issue-1-09/social-studies/strategies-for-preparing- preservice-social-studies-teachers-to-integrate-technology-effectively-models-and-practices/ Burgoyne, N., Graham, C., & Sudweeks, R. (2010). The validation of an instrument measuring TPACK. In D. Gibson, & B. Dodge (Eds.). Proceedings of Society for Information Technology & Teacher Education International Conference, Chesapeake, VA, 3787–3794. https://www.learntechlib.org/p/33971/ Cavanagh, R. F., & Koehler, M. J. (2013). A turn toward specifying validity criteria in the measurement of technological pedagogical content knowledge (TPACK). Journal of Research on Technology in Education, 46(2), 129–148. https://doi.org/10.1080/15391523.2013.10782616 Chai, C., Koh, J., & Tsai, C. (2016). A review of the quantitative measures of technological pedagogical content knowledge (TPACK). In M. C. Herring, M. J. Koehler, & P. I. Mishra (Eds.), Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 87-106). Routledge. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5 Cox, S. (2008). A conceptual analysis of technological pedagogical content knowledge (Doctoral dissertation). Brigham Young University, Provo, UT. https://scholarsarchive.byu.edu/etd/1482/ Creswell, J. W. (2008). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (3rd ed.). Person Merrill Prentice Hall. Cuban, L. (2001). Oversold and underused: Computers in the classroom. Harvard University Press. Delgado, A. J., Wardlow, L., McKnight, K., & O’Malley, K. (2015). Educational technology: A review of the integration, resources, and effectiveness of technology in K-12 classrooms. Journal of Information Technology Education: Research, 14, 397–416. https://doi.org/10.28945/2298 Deng, F., Chai, C. S., So, H. J., Qian, Y., & Chen, L. (2017). Examining the validity of the technological pedagogical content knowledge (TPACK) framework for preservice chemistry teachers. Australasian Journal of Educational Technology, 33(3), 1–14. https://doi.org/10.14742/ajet.3508 Ertmer, P. A. (1999). Addressing first- and second-order barriers to change: Strategies for technology integration. Educational Technology Research & Development, 47(4), 47–61. https://doi.org/10.1007/BF02299597 Ertmer, P. A. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research & Development, 53(4), 25–39. https://doi.org/10.1007/BF02504683 Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255–284. https://doi.org/10.1080/15391523.2010.10782551 Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers and Education, 59(2), 423–435. https://doi.org/10.1016/j.compedu.2012.02.001 https://doi.org/10.2190/EC.48.2.e https://doi.org/10.1016/j.compedu.2010.07.009 https://citejournal.org/volume-9/issue-1-09/social-studies/strategies-for-preparing-preservice-social-studies-teachers-to-integrate-technology-effectively-models-and-practices/ https://citejournal.org/volume-9/issue-1-09/social-studies/strategies-for-preparing-preservice-social-studies-teachers-to-integrate-technology-effectively-models-and-practices/ https://www.learntechlib.org/p/33971/ https://doi.org/10.1080/15391523.2013.10782616 https://doi.org/10.1207/S15328007SEM0902_5 https://scholarsarchive.byu.edu/etd/1482/ https://doi.org/10.14742/ajet.3508 https://doi.org/10.1007/BF02299597 https://doi.org/10.1007/BF02504683 https://doi.org/10.1080/15391523.2010.10782551 https://doi.org/10.1016/j.compedu.2012.02.001 Australasian Journal of Educational Technology, 2021, 37(3). 167 Ertmer, P. A., Ottenbreit-Leftwich, A. T., & Tondeur, J. (2015). Teachers' beliefs and uses of technology to support 21st-century teaching and learning. In H. Fives, & M. Gregorie Gill (Eds.), International handbook of research on teachers' beliefs (pp. 403-418). Routledge. Figg, C., & Jaipal, K. (2013). Using TPACK in practice workshops to enable teacher candidates to create professional development workshops that develop tech-enhanced teaching. In R. McBride, & M. Searson (Eds.). Proceedings of Society for Information Technology & Teacher Education International Conference, Chesapeake, VA, 5040–5047. https://www.learntechlib.org/p/48931/ Graham, C. (2011). Theoretical considerations for understanding technological pedagogical content knowledge (TPACK). Computers & Education, 57(3), 1953–1960. https://doi.org/10.1016/j.compedu.2011.04.010 Harris, J., Grandgenett, N., & Hofer, M. (2010). Testing a TPACK-based technology integration assessment rubric. In D. Gibson, & B. Dodge (Eds.). Proceedings of Society for Technology & Teacher Education International Conference, Chesapeake, VA, 3833–3840. https://www.learntechlib.org/p/33978/ Harris, J., & Hofer, M. (2009). Instructional planning activity types as vehicles for curriculum-based TPACK development. In C. D. Maddux (Ed.), Research highlights in technology and teacher education (99–108). Society for Information Technology in Teacher Education. https://scholarworks.wm.edu/cgi/viewcontent.cgi?article=1005&context=bookchapters Harris, J., Mishra, P., & Koehler, M. J. (2009). Teacher’s technological pedagogical content knowledge: Curriculum-based technology integration reframed. Journal of Research on Technology in Education, 41(4), 393–416. http://doi.org/10.1207/ s15326985ep2803_7 Herring, M., Curran, C., Stone, J., Davidson, N., Ahrabi-Fard, I., & Zhbanova, K. (2015). Emerging qualities of effective teaching: Embracing new literacies. The Educational Forum, 79(2), 163–179. https://doi.org/10.1080/00131725.2015.1006405 Hew, K. F., & Brush, T. (2007). Integrating technology into K-12 teaching and learning: Current knowledge gaps and recommendations for future research. Educational Technology Research & Development, 55(3), 223–252. https://doi.org/10.1007/s11423-006-9022-5 Hofer, M., & Grandgenett, N. (2012). TPACK development in teacher education: A longitudinal study of preservice teachers in a secondary M.A.Ed. program. Journal of Research on Technology in Education, 45(1), 83–106. https://doi.org/10.1080/15391523.2012.10782598 Hofer, M., & Harris, J. (2010). Differentiating TPACK development: Using learning activity types with inservice and preservice teachers. In C. D. Maddux, D. Gibson, & B. Dodge (Eds.), Research highlights in technology and teacher education 2010 (pp. 295–302). Society for Information Technology and Teacher Education. https://scholarworks.wm.edu/cgi/viewcontent.cgi?article=1007&context=bookchapters Hofer, M. & Harris, J. (2016). Open educational resources (OERs) for TPACK development. In G. Chamblee, & L. Langub (Eds.). Proceedings of Society for Information Technology & Teacher Education International Conference, Chesapeake, VA, 2872-2877. Association for the Advancement of Computing in Education. https://www.learntechlib.org/p/172102/ Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118 Hsu, P. S. (2016). Examining current beliefs, practices and barriers about technology integration: A case study. TechTrends, 60(1), 30–40. https://doi.org/10.1007/s11528-015-0014-3 Karns, S. J. (2019). Pairing a learning activity types short course with collaborative curriculum design: An approach to impact teachers’ technological pedagogical content knowledge (TPACK). University of Pittsburgh. http://d-scholarship.pitt.edu/36926/ Keith, T. Z. (2019). Multiple regression and beyond: An introduction to multiple regression and structural equation modeling (3rd ed.). Taylor & Francis. Koehler, M. J., & Mishra, P. (2008). Introducing TPCK. In AACTE Committee on Innovation and Technology (Ed.), Handbook of technological pedagogical content knowledge (TPCK). Routledge. Koehler, M. J., Mishra, P., Bouck, E. C., DeSchryver, M., Kereluik, K., Shin, T. S., & Wolf, L. G. (2011). Deep-play: Developing TPACK for 21st century teachers. International Journal of Learning Technology, 6(2), 146–163. https://doi.org/10.1504/IJLT.2011.042646 https://www.learntechlib.org/p/48931/ https://doi.org/10.1016/j.compedu.2011.04.010 https://www.learntechlib.org/p/33978/ https://scholarworks.wm.edu/cgi/viewcontent.cgi?article=1005&context=bookchapters https://doi.org/10.1080/00131725.2015.1006405 https://doi.org/10.1007/s11423-006-9022-5 https://doi.org/10.1080/15391523.2012.10782598 https://scholarworks.wm.edu/cgi/viewcontent.cgi?article=1007&context=bookchapters https://www.learntechlib.org/p/172102/ https://doi.org/10.1080/10705519909540118 https://doi.org/10.1007/s11528-015-0014-3 http://d-scholarship.pitt.edu/36926/ https://doi.org/10.1504/IJLT.2011.042646 Australasian Journal of Educational Technology, 2021, 37(3). 168 Koh, J. H. L., Chai, C. S., Benjamin, W., & Hong, H. Y. (2015). Technological pedagogical content knowledge (TPACK) and design thinking: A framework to support ICT lesson design for 21st century learning. Asia-Pacific Education Researcher, 24(3), 535–543. https://doi.org/10.1007/s40299-015-0237-2 Kopcha, T. J., Ottenbreit-Leftwich, A., Jung, J., & Baser, D. (2014). Examining the TPACK framework through the convergent and discriminant validity of two measures. Computers and Education, 78, 87–96. https://doi.org/10.1016/j.compedu.2014.05.003 Landis, J. R., & Koch, G. G. (1977). An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics, 33(2), 363-374. https://doi.org/10.2307/2529786 Little, T. D. (2013). Longitudinal structural equation modeling. Methodology in the social sciences. The Guilford Press. Little, T. D., Cunningham, W. A., Shahar, G., & Widaman, K. F. (2002). To parcel or not to parcel: Exploring the question, weighing the merits. Structural Equation Modeling, 9(2), 151–173. https://doi.org/10.1207/S15328007SEM0902_1 Mishra, P. (2019). Considering contextual knowledge: The TPACK diagram gets an upgrade. Journal of Digital Learning in Teacher Education, 35(2), 76–78. https://doi.org/10.1080/21532974.2019.1588611 Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054. https://www.tcrecord.org/content.asp?contentid=12516 Mouza, C. (2016). Developing and assessing TPACK among pre-service teachers: A synthesis of research. In M. C. Herring, M. J. Koehler, & P. Mishra (Eds.), Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 169-190). Routledge. Mouza, C., Yang, H., Pan, Y. C., Yilmaz Ozden, S., & Pollock, L. (2017). Resetting educational technology coursework for pre-service teachers: A computational thinking approach to the development of technological pedagogical content knowledge (TPACK). Australasian Journal of Educational Technology, 33(3), 61–76. https://doi.org/10.14742/ajet.3521 Niess, M., van Zee, E. H., & Gillow-Wiles, H. (2010). Knowledge growth in teaching mathematics/science with spreadsheets: Moving PCK to TPACK through online professional development. Journal of Digital Learning in Teacher Education, 27(2), 42–52. https://doi.org/10.1080/21532974.2010.10784657 Overbay, A., Patterson, A. S., Vasu, E. S., & Grable, L. L. (2010). Constructivism and technology use: Findings from the IMPACTing Leadership project. Educational Media International, 47(2), 103–120. https://doi.org/10.1080/09523987.2010.492675 Pamuk, S., Ergun, M., Cakir, R., & Yilmaz, H. B. (2013). Exploring relationships among TPACK components and development of the TPACK instrument. Education and Information Technologies, 20(2), 241–263. https://doi.org/10.1007/s10639-013-9278-4 Phillips, M., & Harris, J. (2018). PCK and TPCK/TPACK : More than etiology. In E. Langran, & J. Borup (Eds.). Proceedings of the Society for Information Technology & Teacher Education International Conference, Washington, DC, 2109–2116. https://www.learntechlib.org/primary/p/182817/ Porras-Hernández, L. H., & Salinas-Amescua, B. (2013). Strengthening TPACK: A broader notion of context and the use of teachers’ narratives to reveal knowledge construction. Journal of Educational Computing Research, 48(2), 223-244. https://doi.org/10.2190/EC.48.2.f R Core Team (2019). R: A language and environment for statistical computing. http://www.r-project.org/ Rosenberg, J. M., & Koehler, M. J. (2015). Context and technological pedagogical content knowledge (TPACK): A systematic review. Journal of Research on Technology in Education, 47(3), 186-210. https://doi.org/10.1080/15391523.2015.1052663 Rosseel, Y. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. https://doi.org/10.18637/jss.v048.i02 Saad, M. M., Barbar, A. M., Abourjeili, S. A. R., & Abourjelli, S. A. R. (2012). Introduction of TPACK-XL a transformative view of ICT-TPCK for building pre-service teacher knowledge base. Turkish Journal of Teacher Education, 1(2), 41–60. http://tujted.com/makale/616 Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.509.4258&rep=rep1&type=pdf https://doi.org/10.1007/s40299-015-0237-2 https://doi.org/10.1016/j.compedu.2014.05.003 https://doi.org/10.2307/2529786 https://doi.org/10.1207/S15328007SEM0902_1 https://doi.org/10.1080/21532974.2019.1588611 https://www.tcrecord.org/content.asp?contentid=12516 https://doi.org/10.14742/ajet.3521 https://doi.org/10.1080/21532974.2010.10784657 https://doi.org/10.1080/09523987.2010.492675 https://doi.org/10.1007/s10639-013-9278-4 https://www.learntechlib.org/primary/p/182817/ https://doi.org/10.2190/EC.48.2.f http://www.r-project.org/ https://doi.org/10.1080/15391523.2015.1052663 http://dx.doi.org/10.18637/jss.v048.i02 http://tujted.com/makale/616 http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.509.4258&rep=rep1&type=pdf Australasian Journal of Educational Technology, 2021, 37(3). 169 Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14. https://doi.org/10.3102/0013189X015002004 Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-23. https://doi.org/10.17763/haer.57.1.j463w79r56455411 So, H., & Kim, B. (2009). Learning about problem based learning: Student teachers integrating technology, pedagogy and content knowledge. Australasian Journal of Educational Technology, 25(1), 101–116. https://doi.org/10.14742/ajet.1183 Specter, P. E. (1994). Using self-report questionnaires in OB research: A comment on the use of a controversial method. Journal of Organizational Behavior, 15(5), 385–392. https://doi.org/10.1002/job.4030150503 Tondeur, J., Braak, J. Van, & Ertmer, P. A. (2017). Understanding the relationship between teachers’ pedagogical beliefs and technology use in education: A systematic review of qualitative evidence. Educational Technology Research and Development, 65(3), 555–575. https://doi.org/10.1007/s11423-016- 9481-2 US Department of Education (2017). Reimagining the role of technology in education: 2017 National Education Technology Plan update. https://tech.ed.gov/files/2017/01/Higher-Ed-NETP.pdf van Buuren, S., & Groothuis-Ousdhoorn, K. (2011). MICE: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67. https://dspace.library.uu.nl/handle/1874/44635 Voogt, J., Fisser, P., Tondeur, J., & van Braak, J. (2016). Using theoretical perspectives in developing an understanding of TPACK. In M. C. Herring, M. J. Koehler, & P. I. Mishra (Eds.), Handbook of technological pedagogical content knowledge (TPACK) for educators (2nd ed., pp. 33-52). Routledge. Yeh, Y., Hsu, Y., Wu, H., Hwang, F. & Lin, T. (2014). Developing and validating technological pedagogical content knowledge-practical (TPACK-practical) through the Delphi survey technique. British Journal of Educational Technology, 45(4), 707–722. https://doi.org/10.1111/bjet.12078 Corresponding author: Daniel Mourlam, daniel.mourlam@usd.edu Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: Mourlam, D., Chesnut, S. R., & Bleecker, H. (2021). Exploring preservice teacher self- reported and enacted TPACK after participating in a learning activity types short course. Australasian Journal of Educational Technology, 37(3), 152-169. https://doi.org/10.14742/ajet.6310 https://doi.org/10.3102%2F0013189X015002004 https://doi.org/10.17763/haer.57.1.j463w79r56455411 https://doi.org/10.14742/ajet.1183 https://doi.org/10.1002/job.4030150503 https://doi.org/10.1007/s11423-016-9481-2 https://doi.org/10.1007/s11423-016-9481-2 https://tech.ed.gov/files/2017/01/Higher-Ed-NETP.pdf https://dspace.library.uu.nl/handle/1874/44635 https://doi.org/10.1111/bjet.12078 file:///C:/Users/daniel.mourlam/Downloads/daniel.mourlam@usd.edu https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.6310