NHSA Dialog, 25(1), Copyright © 2022, ISSN: 1930-9325 Teacher-Level Traits as Moderating Factors of Professional Development Kate Ascetta University of South Carolina Beth Harn University of Oregon This professional development (PD) study investigated the role of teachers’ initial receptive language skills on the implementation of language enhancement strategies. Early childhood lead (n = 12) and assistant (n = 9) teachers received PD designed to increase their use of language enhancement strategies. Results indicated that teachers’ receptive vocabulary scores predicted their pretest language strategy usage; F(1, 19) = 6.40, p < .05, accounting for 25.2% of the variation in the frequency of language enhancing strategy. Additionally, there was a significant difference detected by role at pre-intervention indicating assistants delivered fewer language enhancement strategies compared to lead teachers M = 41.3 (SD = 16.2) vs. M = 25.7 (SD = 12.0). However, post-intervention means demonstrated a closing of the gap between lead and assistant teachers’ implementation of language enhancement strategies. Results interpreted by discussing the importance of examining the moderating impact of teacher-level traits in evaluating the effectiveness of PD. Keywords: professional development, language INTRODUCTION Professional development conducted in educational settings is highly sensitive to contextual factors (e.g., students skills, teachers traits, school type; Desimone & Hill, 2017); factors which may influence the effectiveness of a professional development experience. Desimone and Hill (2017) emphasized that the next steps in PD research should focus on “unpacking the black box” and digging deep into the teachers’ traits that moderate the effectiveness of PD. Factors such as age, educational experiences (i.e., variation in educational levels), years of teaching, and adult literacy levels may influence how a teacher engages with PD (e.g., Baker & Smith, 1999; Desimone & Hill, 2017; Neuman, 1999; Zaslow et al., 2010). For example, Zaslow et al. (2010) explored the factors that may impact PD effectiveness in early childhood settings by reviewing 37 RESEARCH ARTICLE 2 ASCETTA & HARN literacy and language studies. They found a wide range of impact, d = .13 to d = 1.41, of the PD impacting teachers’ use of instructional practices. However, they were unable to determine why there was such variation in effectiveness. Only two of the studies they reviewed (Baker & Smith, 1999; Neuman, 1999) explicitly adapted their interventions based on the contextual factors of participants. Specifically, Baker and Smith (1999) made changes to their PD experience based the teachers’ instructional knowledge related to literacy and noted that they wished they had known more about their teachers’ skills prior to intervention. Neuman (1999) found that their teachers in the “Books Aloud” training had a wide range of skill differences as they classified their participants as “highly trained” to “extremely needy” (p. 294) and so the researchers needed to provide more content specific training on the intervention rather than their more general PD planned experience. Desimone and Hill (2017) demonstrated how teachers’ years of experience and prior knowledge affected their engagement with their science instruction PD for secondary teachers. Their findings revealed that teacher content knowledge (influenced by years’ of teaching and degree major) moderated the impact of the PD in that teachers with more knowledge benefited more from the PD. They postulated that the variation in background knowledge affected (minimized) the overall effect size of the PD experience. Hamre et al. (2012) demonstrated similar findings and where teachers’ prior knowledge related to literacy impacted the effectiveness of a semester long course on literacy and language instruction. More research is needed to explore what factors, specifically teacher-level traits, are relevant when designing and evaluating the impact of PD to ensure greater effectiveness. Variability in Classrooms When we think about teacher-level traits that may influence effectiveness of PD, the majority of research has focused on the mediating or moderating impact of years of teaching (e.g., Fischer et al., 2018; Lee et al., 2015) and educational level (e.g., Kraft & Papay, 2014). Research has consistently demonstrated that novice teachers (i.e., 1-3 years) are more likely to actively participate in PD than more established teachers (Berends, 2000; Feiman-Nemser, 2012). Also, participant age does not necessarily play a significant impact on the adult learners’ interest in or the effectiveness of online learning experiences (Mulenga & Liang, 2008). However, the findings on the impact of educational levels and majors are more inconsistent and less predictive. For example, Williford et al. (2015) reported that teachers who majored in early childhood were more likely than those who did not to identify the occurrence of specific teacher-child interactions. While, Early et al. (2006) found that teachers’ educational levels were not correlated with instructional skills. Research on educational level and teaching experience remains inconclusive – suggesting perhaps that additional factors maybe moderating the effect of PD interventions. Teacher Characteristics The present study examined two lesser researched factors, teacher’s receptive vocabulary and their role (lead vs. assistant) in the classroom, and how those two traits might moderate the effect of language-based professional development intervention. TEACHER LEVEL TRAITS 3 Receptive Vocabulary Early childhood educators are reported to have the lowest adult language and literacy rates when compared to other educators (Halle et al., 2009; Zaslow et al., 2010). Considering the emphasis placed on providing children, particularly those experiencing early disadvantages, with a language- rich environment the possibility of low language levels for early childhood teachers is particularly concerning. Teachers are the primary language models for children in preschool settings. And yet, we currently do not measure the pre-skills our teachers have as part of our PD efforts that may moderate or mediate their ability to provide language-rich environments for children (Ascetta et al., 2019; Halle et al., 2009; Phillips et al., 2003). We have established that teachers’ conceptual knowledge of math and sciences is more likely to influence their ability to improve outcomes for their students (Berends et al., 2002). It is critical then that we build the same understanding of teachers’ language ability and how it might influence the creation of language-rich environments in early childhood settings (Zaslow et al., 2010). Our study examined the teachers’ receptive vocabulary and the possible impact of their skills on a PD experience. Role of Teacher Position Traditionally early childhood classrooms are staffed with two teachers: a lead and an assistant teacher (e.g., Pianta et al., 2005; Sec. 648A, 2021). Staffing responsibilities have shifted over time and we now see assistant teachers providing more instructional support across the day (Garner et al., 2015; Han & Neuharth-Pritchett, 2010). For example, Sosinsky and Gilliam (2011) found that in Head Start classrooms, when compared to K-12 settings, lead and assistant teachers shared much of the same tasks (e.g., leading small instruction groups, reading books, facilitating play) in the daily classroom routine. Bullough (2015) noted that only two studies have explicitly examined teaming in Head Start classrooms (i.e., Fitzgerald & Theilheimer, 2013; Shim et al., 2004); as a field we know very little about the dynamics between lead and assistant teachers. There is great variability in the experiences, both in-service training and education, of lead and assistant teachers in early childhood contexts. Very few studies have specifically examined the educational backgrounds of assistant teachers in early childhood contexts. Assistant teachers tend to have lower education levels compared to lead teachers (Garner et al., 2015; Jones et al., 2012). Unlike lead teachers, who typically have earned a bachelor’s degree (Barnett et al., 2010), it is common for the highest degree held by assistant teachers to be a high school diploma (Bellm et al., 2002). It should be noted that in 2013 Head Start standards started to require all assistant teachers be either have or be enrolled a program leading to at least a CDA (Sec. 648A, 2021); we do not have access to national data regarding the current educational status of Head Start assistant teachers. As previously stated, most early childhood classrooms are staffed with two or more teachers and yet it is common practice in research to only include the lead teacher in studies of professional development (e.g., Sosinksy & Gilliam, 2011). Only a handful of studies (Ascetta et al., 2019; Sosinksy & Gilliam, 2011; Curby et al., 2012) explicitly include assistant teachers. With even fewer specifically targeting supports for early childhood assistant teachers (e.g., Domitrovich et al., 2009). Furthermore, little to no research exists that examines potential differences, in 4 ASCETTA & HARN conceptual knowledge and instructional practices, between lead and assistant early childhood teachers (Curby et al., 2012; Fraser & Meadows, 2008). Early childhood research often disregards the inclusion of assistant teachers when conducting data analysis and fewer have specifically compared the effectiveness of a PD intervention on lead and assistant teachers (Ascetta et al., 2019; Curby et al., 2012). Curby et al. (2012), examined language differences and found statistically significant different implementation rates of instructional supports (i.e., language modeling, quality of feedback, concept development) for the two groups with assistant teachers scoring lower than the lead teachers. They suggested that experience (e.g., years teaching, professional development) might play a larger role in the differences found between the two groups than educational attainment. Further research is needed to isolate the factor(s) that contribute to variation in quality and quantity of language instruction implemented by lead and assistant teachers. Recent studies, including the present one, seek to highlight the need to examine contextual factors when designing, delivering, and examining the effect of PD (e.g., Desimone & Hill, 2017; LoCasale-Crouch et al., 2016). The Present Study This study explored the potential relation between teacher initial receptive vocabulary skills and effect of role on the implementation of language enhancement strategies. This data is part of a larger PD intervention study (please contact first author). All teachers, leads and assistants, received access to the online PD (i.e., language enhancement strategies and self-monitoring content) and individualized feedback. Our study aimed to understand the following exploratory research questions: 1. To what extent do teachers’ receptive vocabulary skills predict their usage of language enhancement strategies? 2. Does teacher role (lead vs. assistant) moderate the frequency of language enhancement strategies implementation? MATERIALS AND METHODS Participants We recruited from Head Start programs in the Pacific Northwestern region. Twelve teaching teams (i.e., lead and assistant teachers) agreed to participate creating a convenience sample. Twelve classrooms, 24 teachers, consented to participate. We had three inclusion criteria: (a) teacher, either assistant or lead, employed in only one Head Start classroom (not a rotating position); (b) work with children ages 3-5 years old; and (c) reliable access to the internet. Please see Table 1 for teacher demographic data. All teachers were female and their average age was 34 (range 22 – 61). The analysis sample included 21 teachers (12 lead teachers and 9 assistant teachers). Three assistant teachers withdrew from the study prior to pretest; due to maternity leave, left position, and no longer wanted to participate. TEACHER LEVEL TRAITS 5 TABLE 1. Demographics For Teachers Lead (n = 12) Assistant (n = 9) Number of years at Head Start <1 2 3 4 5+ 3 4 2 1 2 0 1 3 0 4 Educational background High School GED Associates Bachelors Masters 0 0 3 5 4 2 3 3 1 0 Ethnicity White Asian Hispanic/Latino Other 11 1 0 0 3 0 5 1 PPVT-4 Standard Scores M = 92.83 (87 – 100) M = 83.33 (76 - 89) Procedures All 21 teachers received access to the online learning management system used to deliver the intervention (i.e., language enhancement strategies (LES) and self-monitoring modules). The first author created a module for each of the five language enhancement strategies (i.e., repetition, expansion, open-ended questions, narration, and self-talk) and two additional modules related to self-monitoring. Each LES module showed the clip of a classroom teacher, provided the operational definition of the strategy (see Table 2) and then presented the clip again with the strategy highlighted when the language strategy occurred in the video clip. Teachers were then prompted to view the clip again and see if they could identify the language strategy used. The final aspect of each LES module contained written examples, with voiceover narration, of when teachers’ might use the strategy in their classroom. The two self-monitoring modules included: (1) a brief recorded PowerPoint presentation on what self-monitoring is and why one would use it; and (2) information on how to complete the self-monitoring form. Each module took approximately ten minutes, for a total viewing time of 70 minutes. Teachers were required to view all modules at least one time; they had continued access to all modules during the four weeks that they received feedback on their use of the LES. Only one teacher viewed the modules more than once. After viewing the modules, teachers were prompted to practice the strategies for four weeks. Each week teachers submitted both an online self-monitoring form and a 10-minute video; they submitted a total of four forms plus four videos. The intervention occurred over six weeks. 6 ASCETTA & HARN TABLE 2. Operational Definition and Examples of Language Enhancement Strategies Language Facilitation Strategy Example Repeat: Following a child’s utterance, the teacher provides a repetition within 3 s. This includes (a) repeating words in the child’s utterance. Not required that the child responds. A child states, “car” and the teacher responds, “Yes a car.” Expansion: Following a child’s utterance, the teacher provides an extension within 3 s. This includes adding new descriptive information to the utterance. Not required that the child responds. A child reaches for cookie on the counter saying “cookie” and the adult responds, “You want a cookie.” Open-ended question: Teacher verbally prompts with a question that does not require a one-word response. Expected that a child will verbally responds using at least a two- word utterance. Teacher asks, “What do you think will happen next in the book?” Self-talk: Teacher verbally describes his/her actions. Not required that the child responds. Teacher states, “I’m getting the cd player for music time.” Narration: Teacher verbally describes the actions of a student(s). Not required that the child responds Child building in block area. Teacher states, “You’re putting the blocks on top of each other.” Measures Several measures were used to address the research questions related to understanding the potential moderating effect of teacher level traits. Prior to intervention, we collected a demographic survey (e.g., lead or assistant teacher, age, educational levels, etc.) and assessed their receptive vocabulary skills. Additionally, we collected our outcome measure, the use of LES via video submissions, at three points in time (i.e., pre-intervention, post-intervention, and follow-up). Teacher Receptive Vocabulary The Peabody Picture Vocabulary Test, Fourth Edition (PPVT-4; Dunn & Dunn, 1997) is a norm- referenced measure of receptive language skills. Test-retest reliability for the PPVT-4 is r = .93 and the internal consistency by age is reported as α = .94. The PPVT-4 was administered over Skype or Google Hangout with each teacher following the PPVT-4 -4 tele-practice administration (“Telepractice and the PPVT-4”, 2016). TEACHER LEVEL TRAITS 7 Teacher Language Enhancement Strategies For our outcome measure we collected data on the frequency that teachers implemented the five LES by coding videos. Data was collected on the teachers’ usage of the five strategies taught in the PD: (a) open-ended question, (b) narration, (c) self-talk, (d) expansion, and (e) repetition (Ascetta et al., 2019; Girolametto & Weitzman, 2002; Pianta et al., 2008). See Appendix A for operational definitions and examples of the five LES, which are related to increasing language outcomes (e.g., Girolametto & Weitzman, 2002; Pianta et al., 2008). Data Collection The observational data was gathered from three 10-minute videos during each phase of the study: three at pre-intervention, three immediately after intervention, and three approximately one month after intervention. At each phase of the study, the videos captured the teachers engaging with children during three different daily classroom activities: (1) a mealtime, (2) a structured whole/small group, and (3) a free play activity. The first author trained four data collectors, blinded to the purpose of the study, to code the videos collected. Training included reviewing the language enhancement strategies and practice coding in person. Then each coder received access to six new videos, to determine reliability. The coders achieved an average Kappa of .81 with a range of .76-.95, which is considered strong (McHugh, 2012). Additionally, we calculated drift assessment for 20% of the videos with Kappa calculated at .82 (range .80 - .91), .83 (range .79 - .85), and .82 (range .81 - .83), for pre-intervention, post- intervention, and follow-up. Analytic Plan Three different analytic methods were employed to address our first research question - regarding the predictive nature of teachers’ receptive vocabulary skills (i.e., PPVT-4 standard scores) on their usage of LES. We initially selected hierarchal linear modeling (HLM) due to the nested structure of the data (time nested within teacher) (Raudenbush & Bryk, 2002). Specifically, the use a growth model was justified because data collection occurred at multiple time points (pretest, posttest, and follow-up). Time was coded as: pretest = 0, posttest = 4, and follow-up = 5). After finalizing the unconditional growth model, a set of conditional growth models examined the first research question about the effect of intervention condition (i.e., graphed self-reported vs. performance feedback) on teacher total use of language facilitation strategies across time (i.e., pretest, posttest, and follow-up). A growth model was employed to examine the potential effect of teachers’ roles (i.e., lead or assistant teacher) on their total use of language facilitation strategies across time (i.e., pretest, posttest, and follow-up). The outcome, total use of language facilitation strategies, remained the same as the previous growth model. First, the PPVT-4 standard scores were added as a level two predictor in the two-level HLM growth model. The outcome, total frequency use of language facilitation strategies, remained the same as the previous growth models except that the random effect for the slope (time) was dropped from 8 ASCETTA & HARN this model due the underpowered nature of the study. The HLM software did not allow it to run when level-2 predictors were added. This conditional model introduced PPVT-4 first as predictor of change over time and then the intercept; centered on the grand mean. When we were unable to detect a statistically significant effect, likely due to the underpowered nature of this exploratory study, we decided to conduct some additional exploratory analyses. Next, we ran a linear regression to test the predictive nature of PPVT-4 on teachers’ pre-intervention LES usage. Last, a point-biserial correlation was run to further understand differences in their PPVT-4 scores, such as the possible relation between teachers’ roles) and the PPVT-4 scores. To investigate if the implementation of LES varied between lead and assistant teachers we used two approaches. First, we entered the role as a possible predictor of variance for teachers’ language enhancement frequency totals using an HLM growth model. An additional exploratory analysis, point-bi-serial correlation, was conducted to gain further understanding the possible impact of role on LES. RESULTS Please see Table 3 for means, standard deviations, and sample sizes. The following sections will examine the effect of teachers’ receptive vocabulary and role on their use of LES. TABLE 3. Means, Standard Deviations, and Sample Sizes for Teacher Language Enhancement Strategy Frequency of Usage Lead Assistant Language Enhancement Total n M Range SD n M Range SD Pre-intervention 12 42.8 24-83 17.3 9 26.4 17-46 12.6 Post-intervention 12 68.0 38-121 23.9 9 44.4 29-83 22.4 Follow Up 12 62.7 30-101 21.5 9 39.9 14-69 16.7 Teacher Receptive Vocabulary The teachers’ PPVT-4 standard scores were entered as a level-2 predictor. The outcome measures was total frequency use of language enhancement strategies in our growth models (see Table 4). The random effect for the slope (time) was dropped from this model due the underpowered nature of the study. This conditional model introduced PPVT-4 first as predictor of change over time and then the intercept; centered on the grand mean. The PPVT-4 scores estimated an additional 1.05 language enhancement strategy used at the intercept (pre-intervention), 𝑡(19) = 1.89, 𝑝 = .07. Additionally, teachers’ pre-intervention PPVT-4 scores, on average, effected the change over time in frequency usage by 0.39 points 𝑡(38) = 1.74, 𝑝 = .09. The Pseudo-R2 for predicting the intercept, after adding PPVT-4 in this model was 0.80; thus 80% of pre-intervention (intercept) variance is explained by the PPVT-4. Also, the Pseudo-R2 for predicting the slope, after adding TEACHER LEVEL TRAITS 9 PPVT-4 in this model was 0.82; thus 82% of between-teacher variance is explained by PPVT-4 standard scores. 𝐿𝑒𝑣𝑒𝑙 1: 𝐿𝑎𝑛𝑔𝑆𝑡𝑟𝑎𝑡𝑖𝑗 = 𝛽𝑜𝑗 + 𝛽1𝑗(𝑡𝑖𝑚𝑒𝑖𝑗) + 𝑟𝑖𝑗 𝐿𝑒𝑣𝑒𝑙 2: 𝛽𝑜𝑗 = 𝛾00 + 𝛾01(𝑃𝑃𝑉𝑇4𝑗) + 𝑢𝑜𝑗 𝛽1𝑗 = 𝛾10 + 𝛾11(𝑃𝑃𝑉𝑇4𝑗) We examined the potential associative relation between teachers’ PPVT-4 scores and their pre- intervention LES usage by running a linear regression. A scatterplot of pre-intervention language usage against PPVT-4 scores with superimposed regression line was plotted and visual inspection indicated a linear relation between the variables. The residuals demonstrated homoscedasticity and normality. The PPVT-4 scores significantly predicted teachers’ pre-intervention language enhancement total scores, F(1, 19) = 6.40, p < .05, accounting for 25.2% of the variation in language total scores with a medium effect size, adjusted R2 = 21.3% (Cohen, 2013). The regression equation predicted: pre-intervention language enhancement use = -27.26 + .46 x (PPVT-4 score). Next, a point-biserial correlation was analyzed to explore the potential relation between teachers’ roles and their PPVT-4 scores. Homogeneity of variances for PPVT-4 scores for lead and assistant teachers were assessed using Levene's test for equality of variances (p = .07). We found a statistically significant correlation between teachers’ roles and PPVT-4 scores, rpb(51) = .73, with lead teachers associated with higher PPVT-4 scores than assistants, M = 92.8 (SD = 4.4) vs. M = 83.3 (SD = 5.0). TABLE 4. Conditional Growth Model for Teacher Total Frequency of Language Enhancement Usage with Receptive Vocabulary Scores Fixed effects Random effects Coefficient t df p Estimate Chi-square df p Language Enhancement Intercept 27.76 3.78 19 .001 111.62 33.22 19 .02 PPVT-4 x intercept 1.05 1.89 19 .07 Time 10.65 3.38 38 .002 PPVT-4 x Time 0.39 1.74 38 .09 Role of Teacher We employed a growth model to examine the potential effect of teachers’ roles on their total use of language enhancement strategies across. This conditional model introduced role as both a predictor of both time and the intercept. Assistant teachers used 21.7 fewer total language enhancement strategy uses at, the intercept (pre-intervention), then lead teachers, 𝑡(19) = −3.58, 𝑝 < .05 (see Table 5). The Pseudo-R2 for predicting the intercept, after adding role (assistant or lead) in this model was .70; thus 70% of pre-intervention (intercept) variance is 10 ASCETTA & HARN explained by teachers’ role. Additionally, assistant teachers, on average, used 5.67 less total strategies over time than lead teachers, 𝑡(38) = −1.74, 𝑝 = .09. Also, the Pseudo-R2 for predicting the slope, after adding role in this model was .61; thus 61% of between-teacher variance is explained by role. 𝐿𝑒𝑣𝑒𝑙 1: 𝐿𝑎𝑛𝑔𝑆𝑡𝑟𝑎𝑡𝑖𝑗 = 𝛽𝑜𝑗 + 𝛽1𝑗(𝑡𝑖𝑚𝑒𝑖𝑗) + 𝑟𝑖𝑗 𝐿𝑒𝑣𝑒𝑙 2: 𝛽𝑜𝑗 = 𝛾00 + 𝛾01(𝑟𝑜𝑙𝑒𝑗) + 𝑢𝑜𝑗 𝛽1𝑗 = 𝛾10 + 𝛾11(𝑟𝑜𝑙𝑒𝑗) We conducted a point-biserial correlation to explore a potential association between teachers’ role and their pre-intervention and post-intervention total LES use. Using Levene’s test for equality of variance (p = .91), the data demonstrated a homogeneity of variances. Pre-intervention total LES use was not normally distributed (Shapiro-Wilk’s test p < .05). Due to one extreme outlier and skewed distribution of data, the decision was made to use a non-parametric test (Kendall’s tau b) for the point-biserial correlation. A statistically significant association between role and pre- intervention LES usage, 𝜏𝑏 = -.41, p < .05, demonstrated that assistant teachers used less strategies, M = 25.7 (SD = 12.0), than lead teachers M = 41.3 (SD = 16.2). In addition, an effect size was calculated using Hedges’ g, which suggested that for every one standard deviation of change for assistant teachers, there was an approximately 1.08 standard deviation difference for lead teachers prior to intervention, Hedges’ g = 1.08. During post-intervention, we found that the statistically significant association between role and total LES usage remained, 𝜏𝑏 = -.43, p < .05, with assistant teachers still using less language enhancement strategies, M = 44.4 (SD = 22.4), than lead teachers M = 65.6 (SD = 23.9). However, when we compared post-intervention means the effect size decreased to Hedges’ g = .88, slightly closing the gap between lead and assistant teachers’ implementation of LES. DISCUSSION Understanding the dynamic relation between content (i.e., knowledge, skills, or behaviors targeted), the delivery (i.e., how we present the content), and the context (i.e., participants’ characteristics) may provide us with more well designed and ultimately more effective professional development experiences for teachers (Berkel et al., 2011; Roberts et al., 2014). The next section will discuss how the findings from this study contribute to professional development research within the context of early childhood settings. We will present how this study’s findings fit into existing professional development research – specifically the impact of contextual factors on evaluation of effectiveness and possible implications for future design considerations. Contextual Factors and Implications for Effectiveness We hypothesized that teacher characteristics, such as role and language skills, would moderate the effect of the language focused PD intervention. TEACHER LEVEL TRAITS 11 Role Lead and assistant teachers play important roles in early childhood classrooms (Sosinksy & Gilliam, 2011) and while research has shown that lead and assistant teachers often use different types of language (directive vs. enhancing) with children (Curby et al., 2012; Fraser & Meadows, 2008) little research has examined how the two groups might respond differently to language- based professional development interventions (Curby et al., 2012). Our results add to the work of Curby et al. (2012) which found that lead teachers consistently outperformed assistant teachers after receiving a language intervention; the only other study that specifically compared lead and assistant teachers’ responses to language-focused professional development. We found prior to intervention lead teachers, on average, used 21.69 (p< .05 ) more total LES than assistant teachers; and that classroom role explained 70% of pre-intervention variance in LES usage. So, the majority of variation prior to intervention was explained by whether they were a lead or an assistant teacher. We also found while examining group mean differences that while both groups improved their usage of LES, assistant teachers’ post-intervention average usage increased to M = 39.88, which almost equals the lead teacher pre-intervention usage (M = 42.80). Meaning that after our intervention the average assistant used the same number of strategies as lead teachers did prior to PD. Had we known this prior to conducting our study, we may have designed a PD experience with different content (e.g., more examples, etc.) or more intensive supports (e.g., more specific feedback) for the assistant teachers. Additionally, while not statistically significant, we hypothesize that the interaction between time and role may mean that assistant teachers may have improved less than the lead teachers over time; thus given more time or perhaps an increased level of support during the intervention period that gap may have closed. Future work should consider how role impacts a teacher’s ability to respond and engage with PD (LoCasale-Crouch et al., 2016; Roberts et al., 2014); it is critical that we understand the “who” factor and how that shapes the design of PD. We believe that more research is needed to examine, if based on teacher role, participants would benefit from differentiated professional development experiences –tailored to their levels of prior knowledge and skills. This may mean a more intensive intervention that provides additional coaching sessions (more than our sessions 4) and for a longer period of time (greater than four weeks; e.g., Pianta et al., 2008) for assistant teachers to increase their consistent use of instructional strategies. We may also need future research to consider how teachers’ characteristics (e.g., initial skill, role, etc.) may lead to variation in treatment effects for PD (e.g., Desimone & Hill, 2017; Robert et al., 2014; Schachter et al., 2016). If we had not controlled for role in the classroom we may have watered down the effectiveness of the intervention when looking at both groups together; this level of analysis allowed us to see how two groups of teachers reacted to the intervention giving us a more nuanced picture of the PD effectiveness. Zaslow et al. (2010) reported a wide range of effectiveness for language-based PD interventions in early childhood and the inherent difficulty of parsing out true meaning of effect sizes. This ambiguity, we argue as a result of missing moderation analysis procedures, may ultimately impact the determination of what constitutes an evidence-based practice. Thus, more work is needed in this area to best determine how evidence-based practices related to PD work with different teaching populations. 12 ASCETTA & HARN Language Very little is known about the potential moderating role that teachers’ language skills play in the creation of language-rich environments for young children (Halle et al., 2009; Phillips et al., 2003). We measured both their receptive vocabulary skills (i.e., PPVT-4) as well as their overall LES usage throughout the study phases, a unique focus. We acknowledge that receptive vocabulary skills does not fully capture all language and literacy skills, we argue that one could use this a proxy measure for overall language skills. For example, prior research has shown correlations (ranging .56 to .88) between the PPVT and intelligence measures (e.g., Hodapp & Gerken, 1999). We found that teachers’ receptive vocabulary skills, when measured by the PPVT-4 were surprisingly different. Lead teachers had a mean of 92.75 (average range) and a range from 87 to 100. Assistant teachers had a mean of 83.60 (low average) and a range of 76 to 89. Assistant teachers, on average, were almost one standard deviation below the national norm, with many performing well below average. We are unsure at this point how this may directly impact creating a language-rich environment for young children; we did find that the PPVT-4 scores predicted lower pre-intervention language enhancement use (adjusted R2 = 21.3%) and accounted for 25.2% of variation in their use over time. So, one could extrapolate that teachers who scored lower on the PPVT-4 used fewer LES and that children taught by teachers with lower PPVT-4 standard scores received less opportunities to engage in language enhancing exchanges with their teachers. More research is needed to understand how that may or may not translate to classroom settings with preschool-aged children. We do know from parenting literature that when adult caregivers have lower vocabulary levels we see similarly lower average vocabulary levels in young children (e.g., Hart & Risley, 1995). Little research has examined the predictive nature of teachers’ receptive language skills and how it may impact instructional behaviors. Potentially using a measure like the PPVT-4 could provide some guidance for researchers as we develop PD to think about aligning content with the skills of the PD participants. The teachers’ limited receptive vocabulary skills may have moderated the impact of the intervention and future research should consider the impact of this learner characteristic when designing and evaluating the effectiveness of PD. Receptive language skills could be a variable that moderates responsiveness to interventions (Dane & Schneider, 1998; LoCasale-Crouch et al., 2016). We found that teachers’ (lead and assistant combined) pre-intervention language strategy usage ranged from 14 to 83 and post-intervention ranged from 29 to 121. So, the range actually widens after the intervention. Our findings show that some observed teachers had high rates of language enhancement strategy use at both pre- and post-intervention, so they may not have benefited/needed the PD compared to teachers with fewer skills. Knowing more about the teachers’ skills could help aid us as a field in designing more effective PD experiences. For example, anecdotally, one teacher reported that she was aware of the strategies prior to the intervention, and then implemented the strategies with more purpose after the intervention. When looking at the data for the three highest preforming teachers at pre-intervention, all maintained high levels of strategy use at post-intervention; however, what was noted is that they used a wider range of strategies at post-intervention. We hypothesize that some teachers improved marginally while others gained substantially. We suggest future research begin by re-evaluating the measurement tools used to capture changes in teacher behavior. For example, having behavioral measures designed to capture different types of language development (e.g., MLU, vocabulary, TEACHER LEVEL TRAITS 13 response to questions) may provide more insight on how specific language enhancement strategies (i.e., open-ended, narration, self-talk, expansion, extension) impact language development in better, or worse, ways for young children. Also, future research might consider the inclusion of additional coding schemes for adult language – specifically coding schemes that capture instructional practices when teachers are not implementing the LES. One might hypothesize that a moderating factor, not explicitly addressed in this study but observed anecdotally, of interventions could be teachers’ pre-intervention expressive language patterns. This could have implications for both the design – what content and how it is delivered as well as the measurement of effectiveness. We theorize that teachers could be categorized into one of four groups based on their pre-intervention differences in characteristics: (1) uses high rates of language enhancement and low rates of directive language, (2) uses low rates of language enhancement and high rates of directive language, (3) uses low rates of both, or (4) high rates of both. So perhaps a more sensitive tool would measure not only frequency, but also code for different types of language instructional practices (i.e., directive and enhancing). Understanding the patterns of teachers’ expressive language use in classrooms (e.g., frequency and qualities) might begin to explain the variation in teachers’ responsiveness to PD (e.g., Desimone & Hill, 2017; Roberts et al., 2014). Also, this type of examination may identify threshold points (higher/lower skilled) for participants that may explain response to the intervention (Harn et al., 2013). The goal of professional development should move beyond increasing teachers’ total use of strategies, but rather, potentially, the variability and purpose with which they use language enhancement strategies to impact student outcomes. LIMITATIONS AND CONCLUSION We have three limitations to the current study. First, the small number of participants limits our ability to discuss generalization. Future studies will look to replicate and expand this work with a larger sample. When working with a larger sample, we would include additional demographic questions that would broaden our contextual understanding of possible variables that can contribute to the effectiveness of language interventions when working with lead and assistant teachers (e.g., number of years as a teaching pair, prior access to PD related to language enhancement, etc.). In addition to expanding beyond this pilot sample (n = 21), we would recommend utilizing a randomized control trial rather than the quasi-experimental design employed in this current study. Introducing a control group would likely strengthen conclusions and increase the generalizability of such findings. Lastly, we have included a limited number of measures to capture teacher’s language usage (i.e., observation and the PPVT-4). We have begun a line of inquiry to examine contextual factors such as the moderating effect of teachers’ role in the classroom and their receptive vocabulary skills on professional development. It is critical that we seek to identify what teacher-level traits may influence teachers’ interactions with a PD experience. This has implications for both designing (pre) and evaluating (post) responses to PD; understanding the components of PD that are linked to change in teacher practices is one of the next big steps in PD research (LoCasale-Crouch et al., 2016). Examination of the dynamic relation between participants’ characteristics and the features of professional 14 ASCETTA & HARN development is essential to strengthening our early childhood workforce, ultimately promoting improved positive long-term outcomes for children. REFERENCES Ascetta, K., Harn, B., & Durán, L. (2019). Comparing Self-Reported and Performance-Based Online Feedback on Early Childhood Teachers’ Implementation of Language Strategies. Early Childhood Education Journal, 1- 13. Baker, S., & Smith, S. (1999). Starting off on the right foot: The influence of four principles of professional development in improving literacy instruction in two kindergarten programs. Learning Disabilities Research & Practice, 14(4), 239-253. Barnett, W. S., Epstein, D. J., Carolan, M. E., Fitzgerald, J., Ackerman, D. J., & Friedman, A. H. (2010). The State of Preschool 2010: State Preschool Yearbook. National Institute for Early Education Research. Rutgers The State University of New Jersey, 73 Easton Avenue, New Brunswick, NJ 08901. Bellm, D., Burton, A., Whitebook, M., Broatch, L., & Young, M. (2002). Inside the Pre-K Classroom: A Study of Staffing and Stability in State-Funded Pre-K Programs. Berends, M. (2000). Teacher-reported effects of New American School designs: Exploring relationships to teacher background and school context. Educational Evaluation and Policy Analysis, 22(1), 65-82. Berends, M., Chun, J., Ikemoto, G. S., Stockly, S., & Briggs, R. J. (2002). Challenges of conflicting school reforms: Effects of New American Schools in a high-poverty district. Rand Corporation. Berkel, C., Mauricio, A., Schoenfelder, E., & Sandler, I. (2011). Putting the pieces together: An integrated model of program implementation. Prevention Science, 12, 22–33. Bullough Jr, R. V. (2015). Teaming and teaching in ECE: Neoliberal reforms, teacher metaphors, and identity in Head Start. Journal of Research in Childhood Education, 29(3), 410-427. Cohen, J. (2013). Statistical power analysis for the behavioral sciences. New York, NY: Routledge. Curby, T.W., Boyer, C., Edwards, T., & Chavez, C. (2012). Assistant teachers in Head Start classrooms: Comparing to and working with lead teachers. Early Education & Development, 23, 640–653. Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: are implementation effects out of control? Clinical psychology review, 18, 23-45. Desimone, L. M., & Hill, K. L. (2017). Inside the black box: Examining mediators and moderators of a middle school science intervention. Educational Evaluation and Policy Analysis, 0162373717697842. Domitrovich, C. E., Gest, S. D., Gill, S., Jones, D., & DeRousie, R. S. (2009). Individual factors associated with professional development training outcomes of the Head Start REDI program. Early education and development, 20(3), 402-430. Dunn, L., & Dunn, L. (1997). Peabody Picture Vocabulary Test—Third Edition. Circle Pines, MN: American Guidance Service. Feiman-Nemser, S. (2012). Beyond solo teaching. Supporting Beginning Teachers, 69, 10-16. Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., ... & McCoy, A. (2018). Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in response to a nationwide science reform. Teaching and Teacher Education, 72, 107-121. Fitzgerald, M. M., & Theilheimer, R. (2013). Moving toward teamwork through professional development activities. Early Childhood Education Journal, 41(2), 103-113. Fraser, C., & Meadows, S. (2008) Children’s views of teaching assistants in primary school. Education 3–13, 36, 351– 363. Girolametto, L., & Weitzman, E. (2002). Responsiveness of child care providers in interactions with toddlers and preschoolers. Language, Speech, and Hearing Services in Schools, 33, 268–81. Grissom, R. J., & Kim, J. J. (2005). Effect sizes for research: Univariate and multivariate applications. New York, NY: Routledge. Halle, T., Forry, N., Hair, E., Perper, K., Wandner, L., Wessel, J., & Vick, J. (2009). Disparities in Early Learning and Development: Lessons from the Early Childhood Longitudinal Study – Birth Cohort (ECLS-B). Washington, DC: Child Trends. TEACHER LEVEL TRAITS 15 Hamre, B. K., Pianta, R. C., Mashburn, A. J., & Downer, J. T. (2012). Promoting young children's social competence through the preschool PATHS curriculum and MyTeachingPartner professional development resources. Early Education & Development, 23(6), 809-832. Han, J., & Neuharth-Pritchett, S. (2010). Beliefs about classroom practices and teachers' education level: An examination of developmentally appropriate and inappropriate beliefs in early childhood classrooms. Journal of Early Childhood Teacher Education, 31(4), 307-321. Harn, B., Parisi, D., & Stoolmiller, M. (2013). Balancing fidelity with flexibility and fit: What do we really know about fidelity of implementation in schools?. Exceptional Children, 79, 181-193. Hart, B., & Risley, T.R. (1995). Meaningful differences in the everyday experience of young American children. Baltimore: Brookes. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press. Hodapp, A. F., & Gerken, K. C. (1999). Correlations between scores for Peabody Picture Vocabulary Test—III and the Wechsler Intelligence Scale for Children—III. Psychological Reports, 84, 1139-1142. Lee, F. L. M., Yeung, A. S., Tracey, D., & Barker, K. (2015). Inclusion of children with special needs in early childhood education: What teacher characteristics matter. Topics in Early Childhood Special Education, 35(2), 79-88. LoCasale-Crouch, J., Hamre, B., Roberts, A., & Neesen, K. (2016). If you build it, will they come? Predictors of teachers’ participation in and satisfaction with the Effective Classroom Interactions online courses. The International Review of Research in Open and Distributed Learning, 17(1). McHugh, M.L. (2012). Interrater reliability: the kappa statistic. Biochemia medica, 22, 276-282. Mulenga, D., & Liang, J.S. (2008). Motivations for older adults’ participation in distance education: A study at the National Open University of Taiwan. International Journal of Lifelong Education, 27, 289-314. Neuman, S. B. (1999). Books make a difference: A study of access to literacy. Reading Research Quarterly, 34(3), 286-311. Kraft, M. A., & Papay, J. P. (2014). Can professional environments in schools promote teacher development? Explaining heterogeneity in returns to teaching experience. Educational evaluation and policy analysis, 36(4), 476-500. Phillips, D., Crowell, N., Whitebook, M. and Bellm, D. 2003. English literacy levels of the early care and education workforce: A profile and associations with quality of care. Berkeley, CA: Center for the Study of Child Care Employment. Pianta, R., Howes, C., Burchinal, M., Bryant, D., Clifford, R., Early, D., & Barbarin, O. (2005). Features of pre- kindergarten programs, classrooms, and teachers: Do they predict observed classroom quality and child- teacher interactions? Applied developmental science, 9(3), 144-159. Pianta, R. C., Mashburn, A. J., Downer, J. T., Hamre, B. K., & Justice, L. (2008). Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly, 23, 431–451. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Sage. Roberts, A. M., LoCasale-Crouch, J., DeCoster, J., Hamre, B. K., Downer, J. T., Williford, A. P., & Pianta, R. C. (2014). Individual and contextual factors associated with pre- kindergarten teachers: Responsiveness to the MyTeachingPartner coaching intervention. Prevention Science, 16, 1044–1053. Sec. 648A. Staff Qualifications and Development [42 U.S.C 9843a]. (2021). Retrieved from https://eclkc.ohs.acf.hhs.gov/policy/head-start-act/sec-648a-staff-qualifications- development Schachter, R. E., Spear, C. F., Piasta, S. B., Justice, L. M., & Logan, J. A. (2016). Early childhood educators' knowledge, beliefs, education, experiences, and children's language-and literacy-learning opportunities: What is the connection?. Early Childhood Research Quarterly, 36, 281-294. Shim, J., Hestenes, L., & Cassidy, D. (2004). Teacher structure and child care quality in preschool classrooms. Journal of Research in Childhood Education, 19(2), 143-157. Sosinsky, L. S., & Gilliam, W. S. (2011). Assistant teachers in prekindergarten programs: What roles do lead teachers feel assistants play in classroom management and teaching? Early Education and Development, 22, 676– 706. Telepractice and the PPVT-4. (2017, March 12) Retrieved from http://www.pearsonclinical.com/therapy/RelatedInfo/telepractice/telepractice-and-the-ppvt-4.html, 2016 16 ASCETTA & HARN Williford, A. P., Wolcott, C. S., Whittaker, J. V., & Locasale-Crouch, J. (2015). Program and teacher characteristics predicting the implementation of Banking Time with preschoolers who display disruptive behaviors. Prevention Science, 16(8), 1054-1063. Zaslow, M., Tout, K., Halle, T., Whittaker, J. V., & Lavelle, B. (2010). Toward the identification of features of effective professional development for early educators, Literature review. Washington, DC: U.S. Department of Education, Office of Planning. Retrieved from www.ed.gov/about/offices/ list/opepd/ppss/reports.html