Journal of Childhood, Education & Society Volume 1, Issue 2, 2020, pp. 216-228 ISSN: 2717-638X DOI: 10.37291/2717638X.20201248 Research Article ©2020 Journal of Childhood, Education & Society. This is an open access article under the CC BY- NC- ND license. The impact of the quality of early mathematics instruction on mathematics achievement outcomes Bilge Cerezci1 Abstract: The examination of teaching quality in mathematics in relation to student learning outcomes has become increasingly important following the research reports indicating that early mathematics teaching and learning experiences are critical contributors to students’ learning and later achievement in mathematics and other content areas. The purpose of this study is to investigate the relationship between the quality of early mathematics instruction and students’ mathematics learning outcomes in 73 Pre-K to 3rd grade classrooms in an urban public schools system. The results suggested that the quality mathematics instruction varies across observed classrooms but mostly mediocre. Limited but significant associations between instructional quality and mathematics achievement were also documented at the classroom level. More specifically, there was a positive significant interaction between quality of mathematics teaching and students’ mathematics achievement at the end of the school year in classrooms where ratings of the instructional quality was identified as “high,” after controlling for students’ pre-test scores and gender. Article History Received: 03 June 2020 Accepted: 20 July 2020 Keywords Early mathematics; Instructional quality; Mathematics achievement Introduction In today’s global knowledge economy, mathematics proficiency is critically important for all members of the society to achieve, as it constitutes the core of any productive economy. For our society to develop citizens who are knowledgeable and globally competitive, it is essential to provide them with excellent quality mathematical experiences to facilitate their math abilities (Ritchie & Bates, 2013; Watts, Duncan, Siegler, & Davis-Kean, 2014). Mounting evidence indicates the dependence of later school performance on the quality of early math experience (Aunola, Leskinen, Lerkkanen, & Nurmi 2004; Carr, Peters, & Young-Loveridge, 1994; Duncan, et. al., 2007; Lange, Brenneman, & Mano, 2019). If a student falls behind mathematically during the critical years of early schooling, it becomes increasingly unlikely that the student will catch up as he moves up the grade levels (Aunola et. al., 2004; Bodovski & Farkas, 2007). This discrepancy may exist because many students are not developing foundational mathematics knowledge, skills, and confidence needed for success during elementary schooling (Gervasoni & Perry, 2017). Moreover, research suggests that students’ struggles in elementary mathematics are related to weaknesses in early number competence, a fundamental early mathematics concept (Gersten, Jordan, & Flojo, 2005; National Research Council, 2009). Such research results are both distressing and indicative: early mathematics education is foundational and attention to early math education and instruction is vital to improving students’ performance in mathematics. Background The beginning of this century saw the development of two position statements on early childhood mathematics, which set the precedent for the myriad of studies that document the implications of early childhood mathematics knowledge. The first released in the US by the National Association for the Education of Young Children (NAEYC) and the National Council of Teachers of Mathematics (NCTM). _____________ 1 National Louis University, National College of Education, Early Childhood Education, Chicago, USA, e-mail:bcerezci@nl.edu, ORCID: http://orcid.org/0000-0002-9648-2387 https://doi.org/10.37291/2717638X.20201248 https://creativecommons.org/licenses/by-nc-nd/4.0/ The impact of the quality of early mathematics instruction… 217 The National Council of Teachers of Mathematics and the National Association for the Education of Young Children affirm that high quality, challenging, and accessible mathematics education for three-to-six-year-old children is a vital foundation for future mathematics learning. In every early childhood setting, children should experience effective, research based curriculum and teaching practices. Such high-quality practice in turn requires policies, organizational supports, and adequate resources that enable teachers to do this challenging and important work (NAEYC & NCTM 2002/2010, p.1). Following the footsteps of NAEYC and NCTM, the Early Childhood Australia (ECA) and the Australian Association of Mathematics Teachers (AAMT) put forward the following statement: The Australian Association of Mathematics Teachers and Early Childhood Australia believe that all children in their early childhood years are capable of accessing powerful mathematical ideas that are both relevant to their current lives and form a critical foundation for their future mathematics and other learning. Children should be given the opportunity to access these ideas through high quality child-centered activities in their homes, communities, prior to- school settings and schools (AAMT & ECA, 2006, p.1). Both of these statements reiterate the importance of early childhood mathematics education and urge making high quality mathematics teaching and learning a shared experience for all students. These statements also suggest that providing mathematics instruction as early as possible may be particularly beneficial if the early childhood teachers guide children’s mathematical thinking and learning through intentional and explicit teaching. In addition to these leading institutions efforts to turn their nation’s attention to implications of early mathematics teaching and learning, two major research developments have led to growing appreciation of the importance of early math instruction at the global level. Research has suggested that early mathematics performance significantly influences: (1) overall school achievement in mathematics and later life (Aunio & Niemivitra, 2010; Aunola, et. al., 2004), and (2) other subject areas (Carmichael, MacDonald, & McFarland-Piazza 2014; Geary, Hoard & Hamso, 2013). First line of research indicated that children’s mathematical competences differ considerably in early childhood years (Anders & Rossbach, 2012; Sonnenschein & Galindo 2015) and achievement in early mathematics has a profound impact on later success in mathematics (Aunola et al., 2004). A longitudinal study by Aunio & Niemivitra (2010) with 212 kindergarten children suggested that specific mathematics skills such as counting in kindergarten are associated with learning basic and applied arithmetic skills and the overall quality of mathematics achievement in the first grade. Another study done by Aunola and his colleagues investigated how children’s mathematics development occurs from Pre-K to Grade 2. The results suggested that differences among children’s math performance increase over time and these discrepancies exist as early as preschool years (Aunola et al., 2004). Further, consistent with previous studies, recent literature also suggests that early number competence is a strong predictor of future mathematics and school success. For example, Jordan, Kaplan, Ramineni and Locuniak’s (2009) study of early mathematics suggests that early number competence in kindergarten predicts rate of growth in mathematics achievement (between Grade 1 and Grade 3) and achievement levels in Grade 3. The impact of early math skills is not limited to academic achievement in primary grades but carries on through high school and beyond (Duncan & Magnuson, 2011; 2011; Entwisle & Alexander, 1990; National Research Council, 2009; Stevenson & Newman, 1986). For example, Duncan and Magnuson (2011) examined the mathematics achievement of children who consistently exhibited persistent problems in understanding mathematics in elementary school and analyzed it in comparison to children who had stronger early math abilities. The results of the study revealed that 13% of the children with persistent problems were less likely to graduate from high school and 29% of them are less likely to attend college than those who had stronger early mathematics abilities. In other words, the initial differences in mathematics skills in early years may lead children to lag behind their more knowledgeable peers not only in primary grades but also throughout their formal schooling (Geary et al., 1999). Second line of research studies showed the predictive power of early math skills compared to other academic skills, such as reading (Duncan et al., 2007; Duncan & Magnuson, 2011). Lerkkanen, Rasku- Puttonen, Aunola and Nurmi (2005) investigated the relationship between mathematical performance and Bilge CEREZCI 218 reading comprehension among 114 children during the first and second years of primary school. The results indicated that early mathematics skills predict not only later achievement in mathematics but also later reading achievement. Similarly, Duncan and colleagues (2007) conducted a meta-analysis of six large- scale longitudinal data sets to examine the relationship between early learning and later school achievement. Their meta-analysis revealed that early math skills were consistently a stronger predictor of later achievement compared to reading and level of attention (Duncan et al., 2007). Consistent with the educational attainment analyses (Duncan & Magnuson, 2011), early math achievement found to be the most powerful predictor of later school achievement (Duncan et al., 2007). Studies mentioned above have become the cornerstone of the growing movement among researchers, early childhood educators, and policy makers to better understand mathematics instruction in early childhood years (Barnett, 2008; Clements, Sarama & DiBiase, 2004; National Research Council, 2009). In a contemporary society where universal early childhood education has become a reality, the focus of attention has shifted from the availability of early childhood education to its quality. How can we provide our youngsters with the necessary skills and knowledge to succeed in math? How can we ensure that high quality early mathematics teaching and learning experiences is a norm for all children? Examining early mathematics teaching practices in more detail and depth by using measurements of instructional quality at the classroom level, based on the activities that the students and teachers are engaged in during early math lessons, can be a first crucial step in finding an answer to these questions. The Current Study Quality of instruction and students’ instructional experiences in early mathematics lay the foundation for the formal systems of mathematics that will be taught later in school. Despite its importance, our knowledge about quality of mathematics instruction in relation to student achievement outcomes is quite limited (Ball & Rowan, 2004; Brenneman et al., 2011). In order to address this gap in the literature, current study investigates to what extent the quality of mathematics teaching predicts children’s mathematical gains over the course of an academic year in Pre-K to 3rd grade classrooms. Early mathematics instruction is documented, analyzed and assessed according to specific indicators in several areas: the quality of early mathematics content, the quality of implementation, and the extent to which the instruction is facilitated developmentally appropriate learning experiences. Examining the quality of early mathematics instruction through these lenses may shed light on the characteristics of early childhood teachers’ mathematics teaching and how to better support young children’s developing mathematics understanding. Method Sample Description Teacher participants. 73 early childhood teachers (Pre-K to 3rd grade) from 8 public schools in a large Midwestern city in the U.S. participated. Descriptive analysis of the available data about the sample revealed that the number of teachers from each school ranged from 3 to 12. There are 37 teachers from the primary grades (e.g., first, second, and third) and 36 teachers from pre-kindergartens and kindergarten (see Table 1). Table 1. Distribution of teachers by grade level Grade n % Pre-K 15 20.5 K 21 28.8 1st 11 15.1 2nd 12 16.4 3rd 13 17.8 1-2 1 1.4 Total 73 100 Note: N= 73 1-2: 1st and 2nd graders mixed class The impact of the quality of early mathematics instruction… 219 Student Participants. 546 students (Pre-K to 3rd grade) participated this study. Recruited students were included in this study, if they were also; a) enrolled in the classroom of the participating teacher, b) able to complete the student assessments in English or Spanish, c) 4-years-old or older by the time they were first assessed at the beginning of the academic year. Table 2 shows the overall child-level data in terms of their grade level. Table 2. Distribution of students by grade level Pre-K K 1st 2nd 3rd Total n 131 160 89 75 91 546 % 24 29.3 16.3 13.7 16.7 100 Instruments Assessment tools that were used in this study include HIS-EM (The Early Math Collaborative, 2011), WJ-AP subtest (Woodcock, McGrew, & Mather, 2011) and “About my Teaching” on-line survey. High-Impact Strategies for Early Mathematics (HIS-EM) is a lesson-based observation tool designed to be used in Pre-K through 3rd grade classrooms in order to measure the quality of mathematics teaching (The Early Math Collaborative, 2011). As an observation tool, HIS-EM focuses on the intentional instructional activities of a teacher; for that reason, the period of observation is from the start to the finish of a single teacher-directed mathematics lesson. HIS-EM categorizes indicators of quality of early mathematics teaching first according to three primary domains: (1) foundational knowledge of the mathematics (i.e., what); (2) understanding knowledge of young children’s learning in mathematics (i.e., who); and (3) effective use of instructional support in mathematics (i.e., how). Three dimensions further define each domain. “What” domain is assessed through evaluation of the clarity of the learning objectives, the use of math representations and promoting mathematical concept development. “Who” domain includes teacher’s attention to developmental trajectories, response students’ individual needs and use of developmentally appropriate learning formats. The third HIS-EM domain, “How,” reflects the teacher’s ability to plan well organized math lessons, facilitate student engagement and establish math learning communities during the course of mathematics instruction. The HIS-EM measures the extent to which these dimensions of quality teaching practices in early mathematics, both individually and collectively, are present in an observed lesson. Scores are assigned (to each dimension) on a 7-points Likert scale categorized by low (1, 2), medium (3, 4, 5), and high (6, 7) quality ratings. The construct validity of the HIS-EM established through an extensive literature review and consultation with experts in early math education. Reference to the NCTM standards and principles (2000) guided selection of the indicators for the HIS-EM. High levels of inter-rater reliability (.88) and internal consistencies (Cronbach’s alpha .97) have been reported (Cerezci, 2020). The Woodcock-Johnson-III Applied Problems subtest (WJ-AP) is an individually administered norm-referenced test that measures skills in analyzing and solving practical math problems with 60 items. It is the 10th subtest of Woodcock Johnson-III (Woodcock et al., 2011). The test administered verbally presents items involving counting, telling time or temperature, and problem solving. Items are ordered in terms of their age-appropriateness. Testing begins with an item corresponding to the subject’s age and is discontinued after 6 consecutive errors. The score is determined by summing the number of correct responses. Internal alpha reliability estimates are reported as .88 to .94 for English speaking children ages 4 to 7 years. About my teaching is an online survey collecting participating teachers’ demographic information and teaching and learning experiences about early mathematics education. The questions included in the survey were aimed to elicit information about a participating teacher’s educational background, experience in participating pre-service and in-service workshops teaching mathematics, as well as his or her experiences working with English Language Learners (ELL). For example, teachers were asked to answers questions such as: How many years have you been teaching?; About how many hours of in-service math Bilge CEREZCI 220 education workshops have you taken in the last two years?; and How many years of experience do you have working with ELL students in a classroom setting? Procedure for Data Collection Classroom observations. Trained observers conducted live in-class observations in the fall (pre-test) and spring (post-test) at each participating classroom only one HIS-EM observation per classroom in spring and fall at each participating classroom collected. Each HIS-EM observation at each time point (spring or fall) is considered a snapshot representing how mathematics instruction may function across a given school year. All classroom observations were scheduled in advance and conducted during the time the teacher allocated to teach mathematics or the mathematics lesson time period. Scheduled observations were not specific to mathematical content (e.g., number and operations or geometry or etc.), or a particular instructional day (e.g., start or end of a weekly math unit). Observers remained in each classroom for the duration of the mathematics lesson. Applied Problems subtest of Woodcock–Johnson Tests of Cognitive Abilities, 3rd ed., (WJ-AP). Young children’s mathematical achievement was assessed via WJ-AP subtest (Woodcock et al., 2011) in the fall (pre-test) and spring (post-test) in each participating classroom. Because only the children whose parents consented to the study could be assessed, the number of students assessed in each classroom was not consistent. However, the total number of children from each classroom never exceeded 10. For example, if more than 10 students gave consent in any given classroom, only 10 students among all the consenting children were randomly selected and assessed. If the number was not more than 10, then all the consenting children were assessed. Data Analyses In order to examine the relationship between quality of mathematics teaching measured by HIS-EM and students’ learning gains in mathematics over a school year, three-level hierarchical linear modeling (HLM) analyses (Raudenbush & Byrk, 2002) was conducted by using the HLM program. In this analysis, students (Level 1) were nested within teachers (Level 2), who were nested within schools (Level 3). Using three-level HLM, relationships between students’ math achievement and quality of mathematics teaching was estimated after controlling for school variations. Results This study examined the degree to which quality teaching can be used as an indicator for student learning outcomes in mathematics. The association between quality teaching in mathematics and students’ learning gains explored by analyzing data from participating teachers and students. The Distribution of HIS-EM and its Domain Scores Results suggested that overall teaching quality was medium level (M=4.19), ranging from 1.67 to 6.78, with a standard deviation of 1.32 (N=73) (see Table 3). Table 3. Descriptive statistics of overall HIS-EM and HIS-EM domains Mean (SD) Minimum Maximum HIS-EM (Average) 4.19 (1.32) 1.67 6.78 What (Foundational Knowledge in Mathematics) 4.22(1.30) 1.67 7.00 Who (Knowledge of Young Children) 4.17 (1.30) 1.67 6.33 How (Effective Use of Instructional Support) 4.18(1.51) 1.33 7.00 Note: N= 73 The impact of the quality of early mathematics instruction… 221 Table 4 provides sample sizes, means, standard deviations, minimum and maximum WJ-AP standardized score at each time point. As suggested by the WJ-AP standardized score, assessed students’ math performance was lower than the national norm (M=100). On average, WJ-AP scores were 95.14 (ranged from 48 to 134) at pre-test and 96.60 (ranged from 49 to 136) at post-test. On average, male students scored higher at both pre-test and post-test compared to female students and Pre-K students scored higher compared to students between kindergarten and 3rd grade (see Table 4). Table 4. Descriptive statistics of students’ mathematical performance at pre-test and post-test by grade level and gender WJ III Applied Problems Pre-test (Standardized Score) WJ III Applied Problems Post-test (Standardized Score) Group N Mean SD Mean SD Pre-K 131 99.47 11.40 99.03 11.51 K 160 95.56 11.53 96.58 12.26 1 89 92.88 13.78 93.51 11.45 2 75 93.37 13.73 95.89 15.17 3 91 91.85 14.57 96.74 14.41 Male 259 95.67 13.29 97.56 12.78 Female 287 94.59 12.72 95.73 12.81 Overall 546 95.14 12.99 96.60 12.85 Note: N=546 Relationship between Quality of Mathematics Instruction and Teaching and Professional Development Experiences Descriptive analyses of the number of years of teaching experience the observed teachers had and the number of math education PD hours they have attended suggested that, on average, observed teachers had 13.7 years of teaching experience, ranging from 1 to 41 years, with a standard deviation of 9.93 (N=73). The number of PD hours teachers attended, on average, was 12.1, ranging from 0 to 80 hours, with a standard deviation of 16.63 (see Table 5). Table 5. Descriptive Statistics of Teaching and Professional Development Experiences Mean (SD) Minimum Maximum The number of years of teaching experience 13.7 (9.93) 1 41 The number of math education PD hours attended 12.1(16.63) 0 80 Regression analysis performed to investigate the relationship between teachers’ teaching and professional development experiences and their mathematics teaching quality as measure by HIS-EM. The results revealed no statistically significant relationship between commonly used indicators of teacher expertise (i.e., number of years of experience and the number of PD hours teachers attended) and scores on the HIS-EM (observational measure of mathematical teaching quality), R2= 0.27, F (2, 70) = .973, p= .383. The Prediction of Students’ Math Outcomes by Teaching Quality In order to examine the relationship between quality of mathematics teaching measured by HIS-EM and students’ learning gains in mathematics over a school year, three-level hierarchical linear modeling (HLM) analyses (Raudenbush & Byrk, 2002) were conducted by using the HLM program. Hierarchical Linear Modeling (HLM) is a type of regression model often used for analyzing education data sets because they tend to include multiple layers of data that are correlated with one another because they share similar traits (Raudenbush & Byrk, 2002). Three-level HLM analysis conducted where students (Level 1) were Bilge CEREZCI 222 nested within teachers (Level 2), who were further nested within schools (Level 3), to explore whether students’ math outcomes (measured by WJ-AP) was predicted by teachers’ quality of teaching in mathematics (measured by HIS-EM), after controlling for school level variances. In this analysis, students’ mathematics learning was Level 1 outcome variable and teachers’ mathematics teaching quality was Level 2 predictor variable. Model testing is completed in two phases; null model (without predictors) and random intercept and slope model (with predictors at Level 1 and Level 2). The null model. This model was run first in order to determine the partitioning of variance among the three levels of analysis. The fully unconditional HLM model for WJ-AP test results at post-test used as outcome in 3-level HLM analysis is represented below: Math Performance at Post-testijk = γ000+ r0jk + u00k + eijk Analysis of this model revealed χ2 (7) = 11.73, p =.109, and ICC was .01, suggesting that there were not any significant differences in the students’ math performance measured by WJ-AP at the school level. Between Level 1 (i.e., student level) and Level 2 (i.e., teacher level), χ2 (65) = 141.01, p<.001, and ICC was .14 suggesting that there were significant differences in students’ math performance between classes (within the same school); about 14% of the variance in students’ math performance indicated by WJ-AP was between classrooms (i.e., teachers), and about 85% of the variance in students’ math performance was between students within a given teacher’s classroom. For this reason, additional predictors to Level 1 and Level 2 were added for further analysis. Specifically, predictors at the teacher level (HIS-EM, Level 2) and student level (pre-test WJ-AP standardized score and students’ gender) were added to different models to explore whether, and to what extent, the mathematics performance at pre-test, students’ gender, and quality of mathematics teaching measured by HIS-EM explains the differences in math performance at post-test. The random intercept and slope model. This model predicts the level 1 intercept on the basis of the other grouping or predictor variables. This model was performed after partitioning the variance among the three levels. The WJ-AP pre-test scores (centered around the group) and students’ gender (coded dichotomously) were entered to this model as Level 1 predictors of math performance at post-test. The three-level HLM analysis for this model was the following: Math Performance at Post-testijk = γ000 + γ100*GENDERijk + γ200*WJ-AP-PREijk + r0jk + u00k + eijk The addition of gender and math performance at pre-test to this model at Level 1 indicates students’ math performance is a function of the mean math performance at post-test in the classroom, plus some effect of gender and math performance at pretest, plus some individual variation. The results indicated that the gender partially significantly predicted the intercept of the level 1 model (r= -1.38, p= .07), suggesting that on average boys scored higher than girls in WJ-AP at both pre- and post-test. Also, the pre- test score significantly predicted the slope of the level 1 model (r= .66, p<.001), suggesting that the higher the pre-test score, the more likely those students performed higher in the post-test as well. In order to further investigate the effects of Level 2 on Level 1 variables, predictors at the Level 2 were added to random intercept and slope model. In this new model was performed in which both WJ-AP pre-test results and students’ gender were kept as predictors of math performance at post-test and mathematics teaching quality measured by HIS-EM added as predictor at Level 2. The three-level 3-level HLM analysis was the following: Math Performance at Post-testijk = γ000 + γ010*HISEMjk + γ100*GENDERijk + γ200*WJ-AP-PREijk + γ210*WJ- AP-PREijk*HISEMjk + r0jk + u00k + eijk The results indicated that the HIS-EM score did not significantly predict the intercept of the level 1 model (r= .559, p= .353), suggesting that HIS-EM did not predict students’ learning in mathematics after controlling for students’ pre-test scores and gender and school level characteristics. Using one standard deviation above the mean represent high quality mathematics teaching (high scores on HIS-EM), one standard deviation below the mean to represent low quality mathematics teaching (low scores on HIS-EM), and mean score as the average quality of mathematics teaching, observed teachers’ HIS-EM scores were The impact of the quality of early mathematics instruction… 223 categorized as high, low and medium and entered to the model to be analyzed in relation to student mathematics achievement. Even though the overall HIS-EM did not predict students’ mathematics learning, the results also suggested that there are varying effects of teachers’ math teaching quality on students’ mathematics learning. More specifically, teachers who scored high on HIS-EM (one standard deviation higher than the overall mean) more likely to have a positive effect on students mathematics learning at the end of the year (r= .15, p=.027). On the other hand, the effect of students’ mathematics performance at the beginning of the school had significantly less effect on their mathematics performance at the end of the school year if they had a teacher who scored average on HIS-EM (r= -.206, p=.001). The negative interaction suggested that while there is a positive relationship between students’ pre-test and post-test performance, medium quality of mathematics teaching decreased this relationship. Similar kinds of significant relationships between students’ math performance and teachers’ math teaching quality were not observed for teachers who score low in HIS-EM (r= .11, p=.16) (see Table 6). Table 6. Descriptive statistics for levels of teaching quality in relation to student mathematics achievement N r SE p High HIS-EM 15 .15 .07 .027** Medium HIS-EM 41 -.20 .06 .001** Low HIS-EM 17 .11 .07 .160 Note: N= 73 **The correlation is significant at .01 level. Conclusion and Discussion This study investigated the relationship between the quality of early mathematics instruction and student mathematics achievement as measured by the WJ-AP subtest. The current study did not reveal a significant prediction of students’ mathematical learning over a year after controlling for the impact of students’ pre-test scores1 and gender.2 The findings of the study also revealed that indirect indicators of teaching experience (i.e., years of teaching and the number of PD hours teachers attended) did not demonstrate any significant association with mathematics teaching quality. However, the results did find mixed effects of teachers’ degree of mathematics teaching quality on students’ mathematics learning. Specifically, overall mathematics-teaching quality in early childhood classrooms as measured was linked to positive child outcomes when the quality of mathematics instruction was identified as “high.” Teaching and Professional Development Experiences The results showed no statistically significant relationship between commonly used indicators of teacher expertise (i.e., number of years of experience and the number of PD hours teachers attended) and scores on the HIS-EM (observational measure of mathematical teaching quality). Existing research has also shown mixed results on this matter. For instance, Rockoff (2004) found that the teaching experience of teachers matters, but only up to a certain point. It is generally true that less experienced teachers are less likely to provide quality instruction compared to teachers who have ten to fifteen years’ experience. This difference begins to disappear after the less experienced teachers taught about four years (Kane, Rockoff & Staiger, 2006; Rivkin, Hanushek & Kain, 2005; Rockoff, 2004). In terms of the relationship between the number of hours teachers participate in professional development in mathematics and higher quality teaching, some found the positive correlations (King & Newmann, 2000) while others reported mixed results (Goldhaber & Brewer, 2000). _____________ 1 Students’ pre-test score was a significant predictor of their post-test test score. In other words, if a student received a high score at pre-test, they were more likely to receive high score at post-test as well, and vice versa. 2 On average, boys received higher scores on mathematics achievement tests as compared to girls on both the pre-test and post-test. Bilge CEREZCI 224 Despite the inconclusive results, the current finding is noteworthy because it indicates that the effects of experience, whether measured in years of teaching or hours of professional development, are complex and their association with quality of early mathematics teaching is not linear, at least for this group of teachers. Even though no one would claim that years of teaching experience or professional development services do not contribute to teachers’ capacity to provide quality of mathematics teaching, lack of associations might imply that teacher education and professional development programs in early mathematics are not well developed to support teachers. Perhaps the content of these programs and services is not staying up on the latest curricular and pedagogical advances in early mathematics teaching, therefore making it less likely for teachers to deliver quality mathematics instruction regardless of their years of teaching. While what teachers know has tremendous impact on students’ learning outcomes (Darling-Hammond & Bransford, 2005), it is important for the field to redesign the content of teacher education programs and in-service professional development to ensure the continuity of quality mathematics teaching experiences for all students. Varying Teaching, Varying Outcomes The present findings indicate that students who scored lower in pre-test tend to score similarly in post-test. Similar trends also observed among the high scoring students. When the student mathematics achievement are further analyzed in relation to instructional quality, such a trend is not observed. In other words, mathematics instruction in early childhood classrooms as measured by HIS-EM does not predict students’ learning in mathematics. However, when observed teachers’ HIS-EM scores were categorized as high, low and medium and examined in relation to students’ learning gains in mathematics, the results of this study revealed three interesting findings. First, there was a positive significant interaction between quality of mathematics teaching and students’ mathematics achievement at the end of the school year in classrooms where ratings of the instructional quality in mathematics was identified as “high,” after controlling for students’ pre-test scores and gender. These findings exemplified the significance of higher quality mathematics instruction in facilitating students’ mathematics learning. Specifically, teachers who simultaneously exhibited (1) understanding of mathematics content, (2) ability to discern the math content based on students’ development and learning, and (3) skills in employing a range of strategies to move students along, were able to facilitate their students’ mathematics learning. Even though the impact of high quality mathematics teaching on students’ learning was rather small and only concerned a subgroup of students, these findings are consistent with other studies indicating the positive effects of high-quality mathematics teaching on mathematics achievement. That is, students of teachers who provide high quality mathematics teaching make more gains in mathematics than their peers in classrooms with lower quality mathematics teaching (Kyriakides & Creemers, 2008; Nye, Konstantopoulos, & Hedges, 2004; Rockoff, 2004). Second, in classrooms where teachers provided average levels of quality mathematics instruction, there was a negative interaction between quality of mathematics teaching as measured by HIS-EM and students’ pre-test and post-test performance. While students’ pre-test performance was predictive of their post-test performance, the strength of this relationship decreased when teachers provided mediocre levels of mathematics teaching. This result, however, should not imply that all mediocre quality mathematics instruction is deleterious for students’ mathematics learning. Rather, it raises an interesting point, which suggests that teachers with average HIS-EM scores may fail to provide consistent level of mathematics teaching and evenly support their students with varying degrees of mathematical abilities. For advanced students, their instructions may not be challenging enough. For students who are behind, adequate support may not be provided. Third, neither positive nor negative interaction was detected between teachers who provided low quality mathematics instruction and their students’ mathematics performance. This finding implies that when teachers failed to; (1) provide students with meaningful mathematics content, (2) provide opportunities for students to engage with and make sense of the mathematics content that is developmentally appropriate, and (3) creating a learning environment conducive to learning mathematics by using effective instructional support in mathematics, no significant relations can be detected between The impact of the quality of early mathematics instruction… 225 mathematics teaching and learning gains in mathematics. It is not clear why there is no link between the lower quality of mathematics instruction and students’ mathematical learning gains. Much remains to be learned about the lower level of quality mathematics teaching and how it affects students’ learning in mathematics. Despite these interesting findings, one lingering question remains unanswered: Why did teachers’ mathematics teaching quality envisioned in the HIS-EM not correspond to student achievement gains? One reason for the lack of the association could be the need for more data about students. There are multiple factors (e.g., parents, tutors, and the availability learning materials, classroom size) affecting students’ learning outcomes besides the quality of instruction (Epstein, 2018; Garcia & Weiss, 2017; Koretz & Hamilton, 2005), and the existence of these influences on students’ learning make it more difficult to test the relationship between the ratings of quality teaching and mathematics achievement (Sass, 2008). Such information about students, which can have a potential effect on their mathematics learning, was not collected in this study. Thus, further multifaceted data about students is needed in order to determine how mathematics teaching quality influences mathematics achievement. Another reason for the lack of relationship between teachers’ instructional practices and mathematics achievement could involve the state of early mathematics teaching in early childhood classrooms. For example, a study of kindergarten classrooms found that a disparity exists between mathematics teaching and students’ abilities: often, teachers spent significant time on mathematics concepts, such as counting and shapes, which most students had already mastered (Engel, Claessens & Finch, 2013). It is a possibility that the majority of the observed teachers’ understanding of their students’ abilities in mathematics and of what they need to learn might be misaligned with their students’ actual abilities and needs. Such misalignment would make it more difficult to test the mathematics teaching quality as measured by HIS-EM in relation to mathematics achievement, because the tool’s framework is based on teachers’ understanding of the mathematics content and ability to introduce math concepts that are aligned with their students’ development and needs through the use of instructional strategies. Last but not least, it is also important to note that the HIS-EM is developed to be used across multiple grade levels. This, inadvertently, might have carried the risk of failing to document the quality of mathematics instruction to its fill extend. While some aspects of early mathematics instruction are likely to be general, cutting across grade levels, other features of instruction are almost certainly grade level-specific, requiring constructs and indicators that are applicable for each grade level that the tool is designed to be used in. Even though some of the HIS-EM constructs are claimed to be appropriate across varied grade levels (Pre-K to 3rd), the indicators listed might not be necessarily indicative of the quality of instruction across these grades. Therefore, the lack of relationship observed between mathematics instruction and mathematics achievement might be unfortunate result of the tool design and its broad range of applicability in Pre-K through 3rd grade. Taken altogether, the results indicated that the interactions between quality of mathematics instruction and the relationship between students’ pre-test and post-test math performance were not consistent with regard to the degree of their teaching quality. When there was a statistical impact of teachers’ instructional quality on students’ learning, the impact was rather small and only concerned a subgroup of students who were taught by high quality teachers. It is also worth noting that no significant relationship was found between low quality mathematics teaching and students’ learning gains in mathematics. Preliminary evidence supporting predictive validity of HIS-EM produced mixed results and made it difficult to capture and reveal clear linkage between quality of mathematics teaching and students’ outcomes in mathematics across the whole sample of students. Limitations and Future Directions Clearly, the results presented here are promising, yet limited. First, the current study involves a limited sample Future research should examine the applicability of these results in wider array of early childhood classrooms with greater diversity at both child and teacher level. Second, lack of significant associations obtained may be a function of the data collection procedures used and decisions made both at Bilge CEREZCI 226 the student and teacher level. This study acknowledges that “standardized achievement tests, in particular, are exceedingly blunt instruments for measuring what students might learn in a given year from a given curriculum” (National Research Council, 2001, p. 479), and standardized test scores do not always reflect students’ actual state of knowledge and abilities (Erlwanger, 1973; Schoenfeld, 1988). It is possible that even though WJ-AP is a standardized and commonly used measure to test students’ math achievement, it only provides a snapshot of student achievement at a particular point in time and with limited content coverage (e.g., restricted topics, usually only number). Using outcome tools that measure students’ mathematics learning in different mathematics content areas might yield stronger and more consistent results. Furthermore, all teacher level data was collected in single-day observations in each teacher’s classroom. Unfortunately, single-day observations may not necessarily reflect teacher practice across the entire school year. Synthesis of multiple observation cycles could reveal the true relationship between quality of mathematics and instruction and student achievement that was unable to be detected in this data set. Last but not least, it is also possible that there may be other contributors to students’ scores that account for additional variance amongst students’ learning gains in mathematics and were not measured either by HIS-EM or the WJ-AP subtest. Future research should also examine how multiple observations within a short timeframe impacts the estimates of quality of mathematics instruction in relation to mathematics achievement. When considered in light of the fields’ substantial attention to issues of quality of early mathematics teaching and how best to promote students’ early mathematics understanding and learning, this study goes a considerable distance in ascertaining which factors indicate the quality of mathematics teaching. Introducing the High Impact Strategies in Early Mathematics (HIS-EM) as a measure to document early mathematics teaching quality represents a beginning contribution to this effort. The vision of mathematics teaching that guided this study is based on Pedagogical Content Knowledge (PCK) framework put forward by Shulman (1986); and claims that for quality mathematics instruction to occur, early childhood teachers need to familiarize themselves with foundational mathematics content (i.e., what) and the ways in which young children learn, specifically in terms of mathematics (i.e., who), and adopt developmentally appropriate teaching strategies to maximize children’s mathematics learning and growth (i.e., how). HIS- EM was designed as an observational measure to document and assess the quality of early mathematics teaching in relation to this vision for mathematics instruction. While the study will help to contribute to the literature on how to measure early mathematics instruction, more research is needed. If low level of mathematics achievement is ever to be interrupted, and if students are to ever have a chance of succeeding in mathematics, observation measures of early mathematics teaching should continue to seek to understand and identify the characteristics early mathematics instruction that lead to high quality teaching and learning experiences in mathematics. Identifying which types of early mathematics instructions are associated with which developmental outcomes and for whom reflects the sophisticated and nuanced understanding of quality mathematics teaching that is needed to serve the diverse needs of students in our classrooms. Declarations Acknowledgements: Not applicable. Competing interests: The author declares that they have no competing interests. Funding: Not applicable. References Anders, Y., & Rossbach, H. G. (2015). Preschool teachers' sensitivity to mathematics in children's play: The influence of math-related school experiences, emotional attitudes, and pedagogical beliefs, Journal of Research in Childhood Education 29(3), 305-322. https://doi.org/10.1080/02568543.2015.1040564 Aunio, P., & Niemivirta, M. (2010). Predicting children's mathematical performance in grade one by early numeracy. Learning and Individual Differences, 20 (5), 427-435. https://doi.org/10.1016/j.lindif.2010.06.003 https://doi.org/10.1080/02568543.2015.1040564 https://doi.org/10.1016/j.lindif.2010.06.003 The impact of the quality of early mathematics instruction… 227 Aunola, K., Leskinen, E., Lerkkanen, M. K., & Nurmi, J. E. (2004). Developmental dynamics of math performance from preschool to grade 2. Journal of Educational Psychology, 96(4), 699-713. https://doi.org/10.1037/0022-0663.96.4.699 Australian Association of Mathematics Teachers (AAMT) and Early Childhood Australia (ECA) (2006). Position paper on early childhood mathematics. Retrieved from www.pa.ash.org.au/scmainc/docs/Reasearch/earlymaths.pdf. Ball, D. L., & Rowan, B. (2004). Introduction: Measuring instruction. Elementary School Journal, 105(1), 3-10. https://doi.org/10.1086/428762 Barnett, W. S. (2008). Preschool education and its lasting effects: Research and policy implications. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved December 1, 2013, from http://epicpolicy.org/publication/preschool- education. Bodovski, K., & Farkas, G. (2007) Mathematics growth in early elementary school: The roles of beginning knowledge, student engagement and instruction. The Elementary School Journal, 108(2), 115-130. https://doi.org/10.1086/525550 Brenneman, K., Boller, K., Atkins-Burnett, S., Stipek, D., Forry, N., Ertle, … Schultz T. (2011). Measuring the quality of early childhood math and science curricula and teaching. In M Zaslow, I. Martinez-Beck, K. Tout & T. Halle (Eds.), Quality Measurement in Early Childhood Settings (pp.77-103). Baltimore Maryland: Paul H. Brookes Publishing Co.. Carmichael, C., MacDonald, A., & McFarland-Piazza, L. (2014). Predictors of numeracy performance in national testing programs: Insights from the longitudinal study of Australian children, British Educational Research Journal, 40(4), 637-659. https://doi.org/10.1002/berj.3104 Carr, M., Peters, S., & Young-Loveridge, J. (1994). Early childhood mathematics - A framework. In J. Neyland (Ed.), Mathematics Education -A Handbook for Teacher, Vol 1, (pp.262-269). Wellington: Wellington College of Education. Cerezci, B. (2020). Mining the gap: Analysis of early mathematics instructional quality in pre-kindergarten classrooms, Early Education and Development. 1-24. https://doi.org/10.1080/10409289.2020.1775438 Clements, D. H., Sarama, J., & DiBiase, A. M. (Eds.). (2004). Engaging young children in mathematics: Standards for early childhood mathematics education. Mahwah, NJ: Lawrence Erlbaum Associates. https://doi.org/10.4324/9781410609236 Darling-Hammond, L., & Bransford, J. (2005). Preparing teachers for a changing world: What teachers should learn and be able to do. San Francisco, CA: Jossey-Bass. Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P.,… Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 43(6), 1428-1446. https://doi.org/10.1037/0012-1649.43.6.1428 Duncan, G.J., Magnuson, K. (2011). The nature and impact of early achievement skills, attention skills, and behavior problems. In Duncan, G.J., Murnane, R.J. (Eds.), Whither opportunity? Rising inequality, schools, and children's life chances (pp. 47–70). New York, NY: Russell Sage Foundation & Spencer Foundation. Engel, M., Claessens, A., & Finch, M.A. (2011). Teaching Students What They Already Know? The (mis)alignment between mathematics instructional content and student knowledge in kindergarten. Educational Evaluation and Policy Analysis, 35, 157 - 178. https://doi.org/10.3102/0162373712461850 Entwisle, D. R., and Alexander, K. L. (1990), Beginning school math competence: Minority and majority comparisons. Child Development, 61(2), 454-471. https://doi.org/10.2307/1131107 Epstein, J. L. (2018). School, family, and community partnerships (4th ed.). Boulder, CO: Westview Press. Retrieved from https://eric.ed.gov/?id=ED586508. https://doi.org/10.4324/9780429493133 Erlwanger, S. H. (1973). Benny's conception of rules and answers in IPI mathematics. Journal of Children's Mathematical Behavior, 1(2), 88-107. Garcia, E., & Weiss, E. (2017). Education inequalities at the school starting gate: Gaps, trends, and strategies to address them. Washington, DC: Economic Policy Institute. Geary, D. C., Hoard, M. K., & Hamson, C. O. (1999). Numerical and arithmetical cognition: Patterns of functions and deficits in children at risk for a mathematical disability. Journal of Experimental Child Psychology, 74(3), 213-239. https://doi.org/10.1006/jecp.1999.2515 Gersten, R., Jordan, N. C., & Flojo, J. R. (2005). Early identification and interventions for students with mathematics difficulties. Journal of Learning Disabilities, 38(4), 293-304. https://doi.org/10.1177/00222194050380040301 Gervasoni A., & Perry, B. (2017). Notice, explore, and talk about mathematics: Making a positive difference for preschool children, families, and educators in Australian communities that experience multiple disadvantages. Advances in Child Development and Behavior, 53, 169-225. https://doi.org/10.1016/bs.acdb.2017.04.001 Goldhaber, D. D., and Brewer, D. J. (2000). Does teacher certification matter? High school teacher certification status and student achievement. Education Evaluation and Policy Analysis, 22(2), 129-45. https://doi.org/10.3102/01623737022002129 Jordan, N. C., Kaplan, D., Ramineni, C., & Locuniak, M. N. (2009). Early math matters: Kindergarten number competence and later https://doi.org/10.1037/0022-0663.96.4.699 https://doi.org/10.1086/428762 https://doi.org/10.1086/525550 https://doi.org/10.1002/berj.3104 https://doi.org/10.1080/10409289.2020.1775438 https://doi.org/10.4324/9781410609236 https://doi.org/10.1037/0012-1649.43.6.1428 https://doi.org/10.3102/0162373712461850 https://doi.org/10.2307/1131107 https://eric.ed.gov/?id=ED586508 https://doi.org/10.4324/9780429493133 https://doi.org/10.1006/jecp.1999.2515 https://doi.org/10.1177/00222194050380040301 https://doi.org/10.1016/bs.acdb.2017.04.001 https://doi.org/10.3102/01623737022002129 Bilge CEREZCI 228 mathematics outcomes. Developmental Psychology, 45(3), 850 - 867. https://doi.org/10.1037/a0014939 Kane, T. J., Rockoff, J. E., & Staiger, D. O. (2006). What does certification tell us about teacher effectiveness? Evidence from New York City (Working Paper 12155). Cambridge, MA: National Bureau of Economic Research. https://doi.org/10.3386/w12155 King, M. B., & Newman, F.M. (2000). Building school capacity through professional development: Conceptual and empirical considerations. The International Journal of Educational Management, 15(2), 86-94. https://doi.org/10.1108/09513540110383818 Koretz, D., & Hamilton, L. S. (2005). Testing for accountability in K-12. In R. L. Brennan (Ed.), Educational measurement (4th ed) (pp.531- 578). Westport, CT: Praeger. Kyriakides, L., & Creemers, B.P.M. (2008). Using a multidimensional approach to measure the impact of classroom level factors upon student achievement: A study testing the validity of the dynamic model. School Effectiveness and School Improvement, 19(2), 183- 205. https://doi.org/10.1080/09243450802047873 Lange, A., Brenneman, K., & Mano, H. (2019). Teaching STEM in the preschool classroom. New York: Teachers College Press. Lerkkanen, M.-K., Rasku-Puttonen, H., Aunola, K., & Nurmi, J.-E. (2005). Mathematical performance predicts progress in reading among 7-year olds. European Journal of Psychology of Education, 20(2), 121-137. https://doi.org/10.1007/BF03173503 National Association for the Education of Young Children (NAEYC) and National Council of Teachers of Mathematics (NCTM). (2002/2010). Early childhood mathematics: Promoting good beginnings. Retrieved from http://www.naeyc.org/about/positions/psmath.asp National Council of Teachers of Mathematics.(2000). Mathematics teaching today. Reston, VA: NCTM. National Research Council (2001). Adding it up: Helping children learn mathematics. Washington, DC: The National Academies Press. National Research Council. (2009). Mathematics in early childhood: Learning paths toward excellence and equity. Washington, DC: National Academy Press. Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237-257. https://doi.org/10.3102/01623737026003237 Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: applications and data analysis methods (2nd ed). Newbury Park, CA: Sage. Ritchie, S. J., & Bates, T. C. (2013). Enduring links from childhood mathematics and reading achievement to adult socioeconomic status. Psychological Science, 24(7), 1301-1308. https://doi.org/10.1177/0956797612466268 Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417-458. https://doi.org/10.1111/j.1468-0262.2005.00584.x Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review Papers and Proceedings, 94(2), 247-252. https://doi.org/10.1257/0002828041302244 Sass, T. (2008). The stability of value-added measures of teacher quality and implications for teacher compensation policy. Brief 4. Washington, DC: National Center for Analysis of Longitudinal Data in Education Research. Schoenfeld, A. (1988). When good teaching leads to bad results: The disasters of "well taught: mathematics classes. Educational Psychologist, 23(2), 145-166. https://doi.org/10.1207/s15326985ep2302_5 Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. Sonnenschein, S., & Galindo, C. (2015). Race/ethnicity and early mathematics skills: Relations between home, classroom, and mathematics achievement. Journal of Educational Research, 108(4), 261-277. https://doi.org/10.1080/00220671.2014.880394 Stevenson, H. W., & Newman, R. S. (1986). Long-term prediction of achievement and attitudes in mathematics and reading. Child Development, 57(3), 646-659. https://doi.org/10.2307/1130343 The Early Math Collaborative. (2011). High impact strategies in early mathematics. Unpublished. Watts, T. W., Duncan, G. J., Siegler, R. S., & Davis-Kean, P. E. (2014). What's past is prologue: Relations between early mathematics knowledge and high school achievement. Educational Researcher, 43(7), 352-360. https://doi.org/10.3102/0013189X14553660 Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson III. Rolling Meadows, IL: Riverside. https://doi.org/10.1037/a0014939 https://doi.org/10.3386/w12155 https://doi.org/10.1108/09513540110383818 https://doi.org/10.1080/09243450802047873 https://doi.org/10.1007/BF03173503 https://doi.org/10.3102/01623737026003237 https://doi.org/10.1177/0956797612466268 https://doi.org/10.1111/j.1468-0262.2005.00584.x https://doi.org/10.1257/0002828041302244 https://doi.org/10.1207/s15326985ep2302_5 https://doi.org/10.1080/00220671.2014.880394 https://doi.org/10.2307/1130343 https://doi.org/10.3102/0013189X14553660 The impact of the quality of early mathematics instruction on mathematics achievement outcomes