Dialog, 17(1) Copyright © 2014, ISSN: 1930-9325 RESEARCH ARTICLE Knowledge of Mathematical Development Survey: Testing the Validity and Reliability of the Survey and Interpreting Its Results Linda M. Platas University of California, Berkeley The Knowledge of Mathematical Development Survey was developed to measure early childhood teachers’ knowledge of early mathematical development. Measurement properties of the instrument were evaluated through multiple phases of development, including two pilot studies and a final study with 346 pre- and inservice preschool teachers across three states. Validity of the instrument was investigated through content and concurrent measures of validity. Methods included cognitive interviews, literature reviews, and interviews with experts in the field, as well as statistical procedures such as analysis of variance between well-defined groups of pre- and inservice teachers. Cronbach’s alpha coefficients were examined for both the total sample and a subset of three cohorts in order to estimate internal consistency reliability. The study provided preliminary evidence for adequate reliability and validity of the Knowledge of Mathematical Development Survey. Findings revealed two predictors of teachers’ knowledge of mathematical development, classroom experience and completion of a course in mathematical development. Potential applications include evaluation of the impact of early childhood education teacher preparation programs and professional development interventions. Mathematical proficiency is an issue at the forefront of education policy development at all levels (National Research Council [NRC], 2009; RAND, 2003). In particular, the predictive power of early math skills on later academic achievement (Duncan et al., 2007; Romano, Babchishin, Pagani, & Kohen, 2010) has recently elicited much interest in the quality of support of mathematical development in early childhood classrooms. Head Start’s revised early learning framework emphasizes the predictive power of early math skills on later academic achievement in multiple domains (Administration for Children and Families [ACF], 2011, p. 16). Accordingly, major policy stakeholders have issued statements concerning the urgency of including effective mathematics instruction in early childhood curricula (NRC, 2001; National Association for the Education of Young Children [NAEYC] & National Council of Teachers of Mathematics [NCTM], 2002). A primary stakeholder in the field of education as a whole, the American Educational Research Association, stressed in its publication, Early Childhood Education: Investing in Quality Makes Sense, that “the best early childhood programs emphasize language, emergent literacy, and early mathematics skills” (2005, p. 4). In addition, Head Start and many state guidelines now require teachers to provide mathematics instruction in the PLATAS classroom, and call upon programs to implement curricula that ensure progress in this domain (ACF, 2011; Daily, Burkhauser & Halle, 2010). In view of these mandates, it is essential that Head Start and other early childhood teachers possess knowledge of children’s mathematical development and the pedagogy needed to support it in their classrooms. Studies have revealed several challenges facing the early childhood education (ECE) field regarding the implementation of such policies: (a) current classroom support for early mathematical development is generally limited or non-existent (Copple, 2004; Klibanoff, Levine, Huttenlocher, Vasilyeva, & Hedges, 2006; Varol et al., 2012); (b) teacher education and professional development programs frequently lack instruction in early mathematical development (Ginsburg, Lee & Boyd, 2008; Sarama, DiBiase, Clements & Spitler, 2004); and (c) the field lacks research-based instruments that can reliably assess teachers’ knowledge and evaluate the effectiveness of education and training in early mathematics (Maxwell, Field, & Clifford, 2006). It has been suggested that a primary explanation for a lack of support for mathematics development in the early childhood classroom is that teachers rarely receive the preservice education necessary to support cognitive development, and that this particularly applies to early mathematical development (NRC, 2001, 2009). While some graduate programs provide courses in mathematical development, many ECE undergraduate programs have few or no requirements in mathematics education, and those at lower levels of education provide virtually no preparation (Ginsburg et al., 2008, NRC, 2009). Inservice ECE professional development had traditionally focused on “developmentally appropriate” curriculum, literacy development, classroom management skills, and the use of play to promote socio-emotional development (Copley & Padron, 1998; Ginsburg et al., 2006; NAEYC, 2009). More recently, the field has focused on intentional teaching and teacher-child interactions, in particular in the domains of socio-emotional and early literacy development (Powell & Diamond, 2012; Sabol & Pianta, 2012). However, some promising inservice programs have emerged that target early math development. The Institute of Education Sciences has funded several experimental studies that provided professional development in early mathematics to teachers already in the field, in the form of extended workshops and classroom support. Results indicated that the sustained professional development was successful not only in increasing the teachers’ involvement in mathematical activities in the classroom, but also in increasing children’s mathematical skills (Clements & Sarama, 2008; Platas, Klein, & Starkey, 2006). Short-term workshops are a common method of providing training to teachers who are otherwise engaged in classroom teaching and have limited time resources. These workshops are generally offered over the course of just a few days. Short-term workshops on early mathematics are few and far between and, unlike the extended workshops, generally do not support sustainable change in knowledge or practice (Copple, 2004; Ginsburg et al., 2008). Further investigation into the contributions of workshops to teachers’ knowledge of mathematical development would contribute to our understanding in the field of professional development. In light of the lack of education and professional development programs that target teachers’ knowledge of mathematical development and pedagogy, it is not surprising that many teachers of young children do not create mathematically rich environments in their classrooms (Copple, 2004; Ginsburg, Inoue & Seo, 1999). While studies that measure mathematical activities in the preschool classroom are rare (Sarama et al., 2004; NRC, 2009), those that do exist suggest that mathematics-related teacher interactions in the classroom are not common. KNOWLEDGE OF MATHEMATICAL DEVELOPMENT SURVEY Ginsburg et al. reported that there was “little evidence in our observations of any adult involvement – explicit teaching or indirect assistance – in children’s mathematical explorations” (p. 97). In an analysis of frequency of math interactions (math talk) between teachers and children in 26 classrooms, Klibanoff et al. (2006) found that counting activities were present in only 69% of the classrooms and number operations in only 15% (p. 65). Results indicated that the quantity of math talk was associated with an increase in children’s math knowledge over the school year (pp. 62-64). Given this correlation, an examination of factors, including teaching experience, that might contribute to teachers’ ability to support mathematical learning would be of interest. This could be particularly useful, as research on the effects of teacher experience on child outcomes is limited (Munoz & Chang, 2007). PURPOSE OF THE STUDY The purpose of this study was to develop and examine the measurement properties of a new instrument for evaluating teachers’ knowledge of mathematical development, the Knowledge of Mathematical Development (KMD) Survey. The study was a preliminary examination of the reliability and validity of the instrument. Reliability was examined for the total sample of 346 participants and across a subset of three cohorts, differing in education and experience. Validity was examined by an analysis of variance in mean scores of the three cohorts and an analysis of proportion correct for each item across cohorts. In addition, predictors including teaching experience, education level, exposure to professional development workshops in early math development, and completion of a course in mathematical development were investigated for their contributions to teachers’ knowledge of children’s mathematical development. DEVELOPMENT OF THE KMD SURVEY Over the past 30 years, researchers have been intensively studying young children’s mathematical development and have come to understand a great deal about the developmental progression of children’s mathematical understanding (Baroody, Lai, & Mix, 2006; Sarama & Clements, 2009). Early mathematical development can be defined as the increasingly complex mathematical constructions and goals that young children develop and pursue in their activities (Saxe, Guberman, & Gearhart, 1987). For example, can children count a row of items before they can count the same set size of items in a circle array? Or, does the arrangement of the items not matter? Teachers must know the answers to these types of questions when creating effective and developmentally appropriate activities in the classroom. KMD Survey Subdomains Because the field of early mathematics is broad, the mathematical domains covered by this instrument are limited to number and operations. These domains are among the most researched topics in the field and, arguably, the most important in young children’s mathematical development (Clements, 2004). The six subdomains of early number and operations included in this study, along with descriptions and examples are listed in Table 1. PLATAS TABLE 1 KMD Survey Sub-domains Sub-domain Description Examples Verbal counting sequence Stating the number words in sequence “I can count to five, one-two- three-four-five!” Counting/Numerosity Determining numerosity of sets “There are five blocks in my tower.” Ordinal number words Stating position in space or time with ordinal number words “I’m first in line and you’re second!” Addition/Subtraction Joining or separation of sets Kate: “I have one cookie. If you give me one, then I’ll have two.” Sue: “But I only have two, and if I give you one, then I’ll only have one!” Division of sets Fair-sharing activities “I have four cookies, one for me, one for you, one for me, one for you… that’s fair!” Written number symbols and words Recognition and production “I’m four years old… see that number, that’s me, I’m a 4!” A brief content description of the subdomains measured by the KMD Survey is provided below. This section provides support for content validity, demonstrating that the items chosen for inclusion represent current knowledge on mathematical development (Anastasi & Urbina, 1997). Verbal counting sequence. Oral recitation of the counting sequence is the verbal expression of the lexical form of the numbers, and an essential foundation for later object counting. Children first learn the lowest numbers, followed by the higher numbers, and eventually are able to continue the sequence from anywhere in the continuum. Teachers need to understand that these beginning verbal counting sequence recitals do not necessarily correspond to an understanding of the cardinal value of the set (Baroody et al., 2006; Fuson, 1988). Counting/Numerosity. Counting is commonly defined as having the following qualities: (a) a specific number word sequence; (b) each object in a set is labeled with one and only one number word; and (c) that the last number word stated also represents the numerosity of the set, otherwise known as the cardinal number (Fuson & Hall, 1983). Additional ways of defining counting (Gelman & Gallistel, 1978) include the order-irrelevance principle (it does not matter in which order items are counted) and the abstraction principle (all items in a set are considered countable “things”). Other important aspects of counting include knowing the KNOWLEDGE OF MATHEMATICAL DEVELOPMENT SURVEY “number after” a particular number, knowing which number is “bigger,” counting existing sets, and being able to create sets by counting (Baroody et al., 2006). Ordinal number words describing position. Knowledge of ordinal position requires an understanding that there is directionality to a set of objects (e.g., bears lined up at a cave entrance), cardinality, and position in space (Clements & Sarama, 2009; Fuson & Hall, 1983). One of the difficulties in learning ordinal position words is that the words themselves do not always relate to the verbal counting sequence. Examples of this are “one” and “first”, and “two” and “second.” Teachers need to understand that children cannot build upon their knowledge of the verbal counting sequence to gain competency with the initial ordinal numbers (i.e., first, second, third, fifth); instead, children must be exposed to the necessary lexical forms in meaningful conversation and activities. Addition and subtraction. Addition involves the joining of two sets. For young children this is generally represented by problems that either join a set of entities to a set the child already possesses (“You have two apples. If I give you three more, how many will you have?”) or represent part-part-whole situations (“We have three girls and four boys in our group. How many children do we have altogether?”). Subtraction for young children usually entails separation activities (“If you have five blocks and you give me two, how many will you have left?”). Developmental studies illustrate that children are generally limited in these problems to those set sizes that they can represent (Baroody et al, 2006; Clements & Sarama, 2009). Division of sets (fair/equal sharing). Like addition and subtraction, children frequently encounter fair-sharing situations in the home and classroom. In particular, sharing limited resources (e.g., cookies during dessert) can require fair-share apportioning. Although research on young children is limited, studies show that children are able to successfully share equally between themselves and another – or “deal” a set of objects to two entities equally (for instance, two puppets) during their last preschool year (Pepper & Hunting, 1998). The classroom provides many social opportunities where fair-sharing skills are essential in continuing successful play interactions. Written number symbols/word recognition and production. Early experiences with written number symbols and words primarily occur in four forms: (1) recognition of printed number symbols; (2) production of written number symbols; (3) recognizing the printed lexical form of the number words; and (4) producing the lexical form of number words (Baroody, 2004). Item Development The rationale for selecting items for the KMD Survey was based on three criteria: (a) supported by independent research and accepted as representative of the developmental progression by experts in the field (Baroody et al., 2006; Clements & Sarama, 2009); (b) common classroom activities; and (c) common activities found in preschool curriculum books. During development, several child mathematics assessment instruments were reviewed (Woodcock-Johnson III: Applied Problems; TEMA 3) and the author consulted with child mathematics assessors who had become very familiar with these instruments during the course of their extensive research PLATAS experience. Several experts in the field who had taken part in the development of state standards were also consulted. The development of the KMD Survey occurred over a multiphase study that consisted of four parts: item development, two pilot studies, and the study reported in this paper. Experts in the field of mathematical development and education were consulted at each of these phases. In order to test the KMD Survey items for clarity, the first pilot study included cognitive interviews with 20 participants from a broad spectrum of experience and education, ranging from a preschool teacher with 24 units in ECE, to a Ph.D. student who had preschool teaching and mathematics assessment experience. This variability helped to ensure that the survey items would be similarly interpreted by a wide spectrum of future participants. Iterative conversations regarding the scope and phrasing of the 30 items indicated that the majority of the items both represented the scope of early number and operations and used commonly understood terminology. Upon the recommendation of participants, teacher- and child-produced phrases that illustrated the task/activity were added to strengthen item clarity. Ten items were dropped due to length, duplication of content, or little classroom application. Information on items that were eliminated or changed is available from the author. The second pilot study consisted of the completion of the refined draft of 20 items by 53 participants. The primary purpose of this second pilot study was to estimate reliability of the revised instrument. The estimates of reliability from the second pilot were good. Cronbach’s alpha was estimated at .82. The study presented in this paper utilized this refined instrument with a sample of 346 pre- and inservice teachers (examples of KMD Survey items may be found in the Appendix) and provided data to answer two research questions: 1. Are the validity and reliability estimates of the KMD Survey sufficient to support its use in research, program planning, and classrooms? 2. What are the contributing factors in teachers’ acquisition of knowledge of early mathematical development? How do level of education, years of experience, professional development workshops, and math development courses influence this knowledge? METHOD Participants Participants included students from four community colleges in the San Francisco Bay Area, three California State Universities, and three Masters’ programs in two states (western and eastern United States). Participant demographics are listed in Table 2. KNOWLEDGE OF MATHEMATICAL DEVELOPMENT SURVEY TABLE 2 Demographics Community Colleges California State Universities Masters’ Programs All Programs Number of Schools 4 3 3 10 Number of Classrooms 7 6 4 17 Number of Participants 135 147 64 346 Ethnicity (proportion) American Indian .03 .02 .02 .02 Asian .18 .22 .07 .18 Black .10 .05 .12 .08 Latino .27 .21 .04 .21 Native Hawaiian .05 .05 .0 .04 White .34 .45 .75 .45 Female (proportion) .87 .90 .95 .90 Age (mean) 27.3 24.1 30.6 26.6 Years of Experience Range (mean) NA 0-10 (2.29) 0-15 (3.45) 0-15 (2.64) Participants were recruited using a stratified purposeful sampling method in order to facilitate the formation of three distinct cohorts, differing in experience, education and exposure to a math development course. This method provided for the capturing of specific desired variables of particular subgroups of interest, allowed for comparisons between these groups, and provided a way to access fairly large groups of participants simultaneously. In the community colleges, only students in beginning core courses were recruited (consequently, they lacked experience as classroom teachers). At the California State Universities, students in both beginning core courses and advanced 3 rd /4 th year courses were recruited. At the Masters’ level, students currently completing a Mathematical Development course or who had taken a Mathematical Development course as a prerequisite were recruited. All three of the Mathematical Development courses surveyed emphasized theories of development and teaching, understanding of young children’s mathematical thinking, and the development of activities based on those theories and that understanding. These courses were three-credit semester classes. For participants without exposure to a math development course, instruction on mathematical development ranged from none to limited textbook discussion and, at the most, six hours of classroom interaction in curriculum and child development courses. Procedures Instructors were contacted via e-mail and provided with an explanation of the study and a request for the author to visit classrooms to recruit participants and administer surveys. In all cases, permission was granted. Participants were given a $10 gift card as an incentive. In all instances, students were assured that the instructor would not be informed regarding participation. Return rate for completed surveys ranged from 89-100% per classroom, for an overall average of 97%. Completion of the surveys took approximately 15 minutes. PLATAS The use of contrasting groups was the means through which concurrent validity was estimated. Evidence supports the idea that higher qualifications, including more mathematics education training, affect teachers’ knowledge of mathematical development (NRC, 2009). Participants were assigned to one of three cohorts. In order to create these three cohorts, demographic information and background in ECE experience, ECE education and completion of early mathematical development courses and professional development was requested at the end of the survey. Descriptions of these cohorts can be found in the concurrent validity section below. Instrument The instrument consisted of 20 items. To help participants perform as successfully as possible, the easiest items (i.e., items that pilot participants were most successful on) were placed at the beginning of the survey. For each item on the KMD Survey, respondents indicated whether Task #1 or Task #2 represented the easier of the two mathematical tasks for young children, or marked “Same” or “Do not know.” While the correct answer was either Task #1 or Task #2, the other two choices were provided to allow for other possible understandings of mathematical development and to reduce chance correct answers (with only two possible answers, the probability of a random correct answer is .5). Some participants did not know the answer to one or more items and so providing them with instructions to choose “Do not know” provided for more accurate data (Dillman, 2007). For those participants who believed the two tasks to be similar in difficulty, the answer “Same” was an appropriate answer. ANALYSIS The following procedures and statistical tests were used to examine the measurement properties of the KMD Survey. Measures of internal consistency, including Cronbach’s alpha coefficients and item-total correlations, were used as estimates of reliability. For dichotomous data, alpha is equivalent to the Kuder-Richardson 20 (KR20) coefficient and therefore applicable to KMD Survey data (Streiner & Norman, 2008, p. 89). These analyses were performed on the entire sample as well as across the subset of cohorts. Content validity was examined through a literature review, interviews with experts in the field, a review of current child mathematics assessments, and cognitive interviews. In addition, experts in early math assessment and standards were called upon during the drafting of all versions of the KMD Survey to examine the content validity of the items. A one-way analysis of variance (ANOVA) was used to examine concurrent validity by comparing the KMD Survey scores of three groups/cohorts by item and total score. These cohorts were drawn from the participant pool to facilitate the measurement of concurrent validity (Hogan, 2003, p. 184). Three dimensions of importance were considered in setting the parameters for each of these cohorts. These were ECE education, ECE teaching experience, and completion of a mathematical development course. These three cohorts are described in the following paragraphs. The first cohort was representative of students just beginning their careers in the ECE field. These beginning community college and California State University ECE students had no KNOWLEDGE OF MATHEMATICAL DEVELOPMENT SURVEY teaching experience, no ECE education and had not completed a mathematical development course. These two sources, community college and university classrooms, were chosen to reduce the confound of institution. One concern was that a latent factor related to overall test-taking abilities might affect the KMD Survey scores differentially between these two groups. However, when comparing these two sub-groups, there was no significant difference between their average KMD Survey scores, t(44) = .904, p = .371. The second cohort was representative of many teachers currently in the field. In many states, 12 ECE units meet preschool teacher qualification requirements, and many of the teachers currently in the field have completed just 12 units (Ginsburg & Ertle, 2008). This cohort included 3 rd and 4 th year university students with two or more years inservice experience and 12 or more units of ECE. Legislation passed in 2007 stated that by 2013, fifty percent of Head Start teachers must possess a Bachelor’s or higher degree in ECE (Improving Head Start for School Readiness Act of 2007), making this cohort particularly representative of teachers in Head Start. The third cohort represents teachers in an M.A. program with two or more years of inservice experience, and enrollment in a mathematical development course. In all instances, participants in this cohort had either completed the entire course, or were nearing completion of the course (in both cases, study of number and operations had been completed). While admittance to a Master’s program could be a confound, almost no two or four year programs offer a full course in mathematical development (i.e., at least five quarter units or three semester units) and thus it would have been difficult to identify undergraduates who had completed such a course. The third cohort represented teachers most likely to have the greatest amount of knowledge of children’s mathematical development due to experience and education. The third cohort was expected to perform significantly better than the second cohort, and both of these groups were expected to perform better than the first cohort. Since not all of the participants were assigned to a cohort (e.g., there were some students in Masters’ programs who had less than two years of experience), the number of participants included in this analysis was reduced from the total participant pool. While the above analyses provided information on the measurement properties of the KMD Survey, they do not provide information regarding the contribution of individual factors to knowledge of mathematical development. Regression analyses can provide a more detailed picture and has the advantage of quantifying the separate effects of teaching experience, level of education, and completion of math development courses and professional development workshops. As noted earlier, education, experience and exposure to the study of mathematical development have been considered most effective in building teachers’ knowledge of children’s mathematical development. In order to measure the effects of these variables, a regression analysis was conducted on all complete cases. To most accurately measure this, the backward method was used. The backward method is generally preferred over forward regression methods, which are influenced by suppressor effects, and may result in the exclusion of predictors that are significant when other variables are held constant (Field, 2009). PLATAS RESULTS Reliability Estimates of reliability were obtained for the subset of participants who were included in the ANOVA analysis (N=144) and for all participants who completed the KMD Survey (N=346). Cronbach’s alpha results were fairly robust when measured for all participants surveyed as well as for the three-cohort subset. The alpha for the three-cohort subset was .808. This was somewhat higher than the alpha resulting from the analysis that included all participants, .776. However, both were within the realm of adequacy in the field of education (Wiersma & Jurs, 2005). Validity The ANOVA F-test comparing mean KMD Survey scores between cohorts was significant, F(2,141) = 24.38, p = <.001. Due to unequal variances between Cohorts 1 and 3 and Cohorts 2 and 3, Tamhane’s T2 post hoc tests were utilized. Analyses using Tamhane’s T2 post hoc criterion for significance indicated that the average score for each cohort differed significantly, increasing as ECE education, experience and enrollment in a mathematical development course increased. The number of participants in each group and the results of this cohort comparison analysis are listed in Table 3. TABLE 3 KMD Survey Means by Cohort Group N Mean SD Cohort 1 p Cohort 2 p Cohort 1 Preservice Enrolled in core ECE course No math dev course 74 10.30 (3.98) Cohort 2 2+ Years experience in-service Jr./Sr. Status in BA ECE/CD Program No math dev course 33 12.00 (2.94) 1.702 0.046 Cohort 3 2 + Years experience in-service Enrolled in M.A. Program Yes Math dev course 37 14.95 (1.78) 4.648 <.001 2.945 <.001 KNOWLEDGE OF MATHEMATICAL DEVELOPMENT SURVEY Mean KMD Survey scores differed significantly between cohorts, providing support for validity of the KMD Survey instrument. As predicted, the third cohort performed significantly better than the second cohort, and in turn both of these groups performed better than the first cohort. While there was a significant difference between Cohorts 1 and 2 with an effect size of d = .48, the largest differences were those between Cohort 3 and the other cohorts, a difference of 4.65 points between Cohorts 1 and 3 (d = 1.51), and 2.95 points between Cohorts 2 and 3 (d = 1.21). Enrollment in a M.A. mathematical development course was associated with a significant increase in the participants’ knowledge of early mathematical development as measured by the KMD Survey. There were no ceiling or floor effects, adding further support in the estimate of validity. In examination of the proportion correct by item across cohorts utilizing Tamhane’s T2 post hoc tests, 16 of the 20 items showed an increased frequency of correct answers as the cohort education, experience and completion of a mathematical development course increased. All but three items indicated a significant increase in proportion correct when comparing Cohort 1 and Cohort 3. These data are reported in Table 4. It is worth examining these three items more closely. The observed group differences for item 4 approached statistical significance (p = .065). Item 14 was a difficult problem comparing an addition to a subtraction problem with a confound of set size. Because of children’s ability to subitize (immediately perceive the numerosity of a small set; Clements & Sarama, 2009), children can more easily solve a subtraction problem in which one object is taken away from a set of three objects than solve an addition problem where two items are added to a set of five items. Item 16 compared the skills needed to produce a small set from a larger set to those needed to count an existing set. Many participants indicated that these were the same skills (however, counting an existing set is easier than producing a set; Baroody et al, 2006). PLATAS TABLE 4 Proportion Correct by Item across Cohorts KMD Item Proportion Correct By Cohort Difference Cohort 1 vs 3 90% CI 1 2 3 1 .75 .91 .92 .17* [.31, .02] 2 .70 .85 .95 .25*** [.38, .11] 3 .80 .91 .97 .17** [.28, .06] 4 .15 .12 .32 .17 [.37, -.01] 5 .61 .73 .89 .28*** [.45, .12] 6 .31 .46 .78 .47*** [.66, .29] 7 .55 .64 .92 .37*** [.52, .21] 8 .67 .73 .89 .22** [.38, .06] 9 .17 .30 .65 .48*** [.67, .28] 10 .33 .27 .70 .37*** [.57, .17] 11 .65 .88 .95 .30*** [.44, .16] 12 .78 .82 .97 .19** [.31, .07] 13 .52 .67 .87 .35*** [.52, .17] 14 .16 .21 .11 -.05 [.09, -.20] 15 .51 .64 .78 .27** [.45, .07] 16 .73 .76 .68 -.05 [.14, -.26] 17 .51 .58 .89 .38*** [.54, .21] 18 .55 .67 .76 .21* [.40, .01] 19 .67 .67 .87 .20* [.36, .03] 20 .71 .85 1.00 .29*** [.40, .18] Note. Tamhane’s 2; one-tailed; CI = confidence interval * p < .05. ** p < .01. ***p < .001. Contributing factors to teachers’ knowledge of mathematical development Level of education, years of teaching experience, exposure to short-term professional development workshops on mathematical development, and completion of a math development course were examined in a regression analysis to estimate their contribution to knowledge of mathematical development as measured by the KMD Survey. The results of the regression analysis are in Table 5. The findings indicate that when years of experience and level of education, as well as both mathematical development courses and short-term workshops were included in the model, only years of experience and completion of a math development course were significant predictors. This model represents 16% of the variance in the sample. Subsequent models that excluded non-significant predictors did not result in greater predictive power. Examination of residual plots, histograms of residuals, Levene’s Test, Durbin-Watson test, variance inflation factors (VIF) and Cook’s Distance all indicate that assumptions of linearity, normality, homogeneity, independence, non-collinearity and influence were sufficiently met. KNOWLEDGE OF MATHEMATICAL DEVELOPMENT SURVEY TABLE 5 Predictors of Knowledge of Mathematical Development Variable B 95% CI Constant 10.71*** [10.25, 11.18] Years of Experience .24** [0.08, 0.40] Education AA/AS -.01 [-1.20, 1.19] Education BA/BS .92 [-.38, 2.23] Education MA/MS .83 [-1.31, 2.97] Math Development Course 2.02*** [.98, 3.05] Math Development Workshop .41 [-1.01, 1.84] R 2 .16 F 9.73 Note. N=319. ** p < .01 ***p < .001 DISCUSSION AND SUMMARY The purpose of this study was to design and examine the measurement properties of a new instrument for evaluating teachers’ knowledge of children’s early mathematical development and investigate contributing factors to that knowledge. The sample of 346 teachers included both pre- and inservice teachers with a variety of classroom and educational experience. The results indicated that reliability estimates, as measured by Cronbach’s alpha, were adequate for both the sample and cohort subset, with the cohort subset producing a slightly improved estimate of internal consistency. These results, combined with the reliability estimates from the earlier pilot study of 53 teachers, showed a trend of adequate internal consistency. Content validity was examined through a literature review, interviews with experts in the field, a review of current child mathematics assessments, and cognitive interviews. Throughout the development of the instrument, experts in early math assessment were consulted to ensure that the items continued to reflect current research. Concurrent validity was estimated in analyses comparing (a) the mean KMD Survey total scores and (b) item correct frequencies across cohorts. These analyses provided evidence for differences in knowledge of mathematical development as measured by the KMD Survey between the three groups as expected. There were no ceiling or floor effects for the mean KMD Survey total scores, adding further evidence of validity. The frequency of correct answer analyses showed a consistent trend of correct answers as education and experience increased across cohorts. Combined with evidence provided in the development phase and pilot studies, the analyses conducted provided preliminary evidence in support of the validity of the KMD Survey. It has frequently been argued that education makes a difference in the knowledge and practices of early childhood teachers (Bowman, 2011). For researchers who study teachers’ mathematical support in the preschool classroom, a belief in the value of such education, specifically that in mathematical development, is particularly robust (Ginsburg & Ertle, 2008; NRC, 2009). As results from the regression analysis illustrated, it would take eight years of experience to equal the effect of the completion of math development course (for every year of experience, the KMD Survey score rises by .24 points; for the completion of a single math development course, the KMD Survey score rises by 2.02 points). This outcome may explain why so many classrooms lack support for mathematical development, as the results suggest that it may take many years of experience to gain sufficient knowledge about mathematical PLATAS development to provide support in the classroom. The completion of a single math development course significantly adds to teachers’ knowledge of mathematical development, resulting in a 19% increase in the KMD Survey score. Results on the difference between exposure to a professional development workshop and completion of a math development course suggest that sustained and deep interaction with the subject of math development is necessary for lasting results. This study has a number of limitations. Reliability and validity estimates were limited to the populations from which samples were drawn. In order to ensure that an instrument is broadly useful, all applicable populations must be sampled. While this study included a substantial number of participants, there were populations that were not included. Because the sampling occurred in colleges and universities, teachers who were in the workforce, but not enrolled in any classes, were not included. Additional validation and reliability studies that include this population are needed. As noted in the rationale for Cohort 3, there was a possible confound of education level. Cohort 2 and 3 differences included enrollment in a mathematical development course and a Master’s program. The assumption was that the mathematical development course was related to the increase in knowledge about mathematical development. However, it is possible that those students who were admitted to a Master’s program were fundamentally different from those who were still enrolled in a Bachelor’s program (although as indicated earlier, this was not true for differences between beginning community college students and enrollees in a Bachelor’s program). It is unlikely that, given the overall lack of emphasis on mathematical learning and teaching in the field as a whole, simply being admitted to a Master’s program increases one’s knowledge about early mathematical development. Future research disentangling these variables would be useful. As detailed in the introduction, the KMD Survey does not measure teachers’ knowledge of mathematical domains other than number and operations. These two number-based subdomains are frequently subsumed under the dimension of “number sense” (Case, 1998; Gersten & Chard, 1999) and these analyses were performed with the assumption that the survey is uni-dimensional. Other important subdomains include geometry, measurement, algebra (patterns), data analysis and probability. Future expansion of the KMD Survey should investigate the dimensionality of the number sense items as well as include these additional subdomains. More importantly, the KMD Survey does not measure knowledge of the pedagogy necessary to support the teaching of early mathematics. Research on the assessment of this essential component of teaching is urgently needed in the ECE field. Finally, as these are preliminary data, more research is needed. This study has provided estimates of reliability and validity applicable only to this limited sample. In the future development of the KMD Survey with broader populations and more subdomains, more thorough investigations into the measurement properties will be needed. Major policy stakeholders in early education in the United States have issued statements concerning the urgency of including mathematics education in the early childhood curriculum (NAEYC & NCTM, 2002). ECE teacher education programs must now begin to address the lack of education and training in this area of curriculum. The use of the KMD Survey could play a part in this enterprise, at multiple levels of the educational system: (a) individual instructors could use the survey to measure the knowledge of students at the beginning and end of a course to assess change in knowledge; (b) ECE programs could survey a cohort of students to inform KNOWLEDGE OF MATHEMATICAL DEVELOPMENT SURVEY the development of a mathematical development course; and (c) the use of the survey in multiple studies with differing interventions could provide useful comparisons across these interventions. This study goes a considerable distance in ascertaining which factors contribute to teachers’ knowledge of mathematical development. While the study will help to contribute to the literature on effective practices, more research is needed. A complete program of research on the quality and impact of ECE preparation in mathematics education would also include an examination of curricula, as well as investigations into the financial, programmatic and systemic constraints on the implementation of effective mathematics education in the early childhood classroom. REFERENCES Administration for Children and Families. (2011). The Head Start Child Development and Early Learning Framework: Promoting Positive Outcomes in Early Childhood Programs Serving Children 3–5 Years Old. Retrieved from http://eclkc.ohs.acf.hhs.gov/hslc/tta- system/teaching/eecd/Assessment/Child%20Outcomes/HS_Revised_Child_Outcomes_Framework(rev- Sept2011).pdf American Educational Research Association. (2005). Early childhood education: Investing in quality makes sense. Research Points, 3(2), 1-4. Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall. Baroody, A. J. (2004). The developmental bases for early childhood number and operations standards. In D. H. Clements, J. Sarama and A. -M. DiBiase (Eds.), Engaging young children in mathematics: Standards for early childhood mathematics (pp. 173-219). Mahwah, NJ: Erlbaum. Baroody, A. J., Lai, M. L., & Mix, K. S. (2006). The development of young children's early number and operation sense and its implications for early childhood education. In B. Spodek & O. Saracho (Eds.), Handbook of research on the education of young children (pp. 187-221). Mahwah, NJ: Erlbaum Bowman, B. T. (2011). Bachelor's degrees are necessary but not sufficient: Preparing teachers to teach young childen. In E. Zigler, W. S. Gilliam & W. S. Barnett (Eds.), The pre-k debates: Current controversies & issues (pp. 54-57). Baltimore, MD: Paul H. Brookes. Case, R. (1998, April). A psychological model of number sense and its development. Paper presented at the annual meeting of the American Educational Research Association, San Diego. Clements, D. H. (2004). Major Themes and Recommendations. In D. H. Clements, J. Sarama and A. -M. DiBiase (Eds.), Engaging young children in mathematics: Standards for early childhood mathematics education (pp. 7-72). Mahwah, NJ: Erlbaum. Clements, D. H., & Sarama, J. (2008). Experimental evaluation of the effects of a research-based preschool mathematics curriculum. American Educational Research Journal, 45, 443-494. Clements, D. H., & Sarama, J. (2009). Learning and teaching early math: The learning trajectories approach. New York: Routledge. Copley, J. V., & Padron, Y. (1998). Preparing teachers of young learners: Professional development of early childhood teachers in mathematics and science. Paper presented at the Forum on Early Childhood Science, Mathematics, and Technology Education, Washington, D.C. Copple, C. E. (2004). Mathematics curriculum in early childhood. In D. H. Clements, J. Sarama and A. -M. DiBiase (Eds.), Engaging young children in mathematics: Standards for early childhood mathematics (pp. 83-87). Mahwah, NJ: Erlbaum. Daily, S., Burkhauser, M., & Halle, T. (2010). A review of school readiness practices in the states: Early learning guidelines and assessments. Early Childhood Highlights, 1(3), 1-12. Retrieved from http://www.childtrends.org/wp-content/uploads/2013/05/2010-14-SchoolReadinessStates.pdf Dillman, D. A. (2007). Mail and internet surveys: The tailored design method. Hoboken, NJ: John Wiley & Sons. Duncan, G., Dowsett, C., Claessens, A., Magnuson, K., Huston, A., Klebanov, P., et al. (2007). School Readiness and Later Achievement. Developmental Psychology, 43, 1428-1446. Field, A. (2009). Discovering statistics using SPSS (2nd ed.). Thousand Oaks, CA: SAGE. Fuson, K. C. (1988). Children's counting and concepts of number. New York: Springer-Verlag. PLATAS Fuson, K. C., & Hall, J. W. (1983). The acquisition of early number word meanings: A conceptual analysis and review. In H. P. Ginsburg (Ed.), The Development of mathematical thinking. New York: Academic Press. Gelman, R., & Gallistel, C. R. (1978). The child's understanding of number. Cambridge, MA: Harvard University Press. Gersten, R., & Chard, D. (1999). Number sense: Rethinking arithmetic instruction for students with mathematical disabilities. The Journal of Special Education, 33(1), 18-28. Ginsburg, H. P., & Ertle, B. (2008). Knowing the mathematics in early childhood mathematics. In O. N. Saracho & B. Spodek (Eds.), Contemporary perspectives on mathematics in early childhood education (pp. 45-66). Charlotte, NC: Information Age. Ginsburg, H. P., Inoue, N., & Seo, K. (1999). Young children doing mathematics: observations of everyday activities. In J. V. Copley (Ed.), Mathematics in the early years. Reston, VA: The National Council of Teachers of Mathematics. Ginsburg, H. P., Kaplan, R., Cannon, J., Cordero, M., Eisenband, J., Galanter, M., & Morgenlander, M. (2006). Helping early childhood educators to teach mathematics. In M. Zaslow & I. Martinez-Beck (Eds.), Critical issues in early childhood professional development (pp. 171-202). Baltimore, MD: Paul H. Brookes. Ginsburg, H. P., Lee, J. S., & Boyd, J. S. (2008). Mathematics education for young children: What it is and how to promote it. Social Policy Report, XXII(I), 3-23 Hogan, T. P. (2003). Psychological testing: A practical introduction. New York: John Wiley & Sons. Improving Head Start for School Readiness Act of 2007, Pub. L. No. 110-134, 121 Stat. 1363. Klibanoff, R. S., Levine, S. C., Huttenlocher, J., Vasilyeva, M., & Hedges, L. (2006). Preschool children's mathematical knowledge: The effect of teacher "math talk". Developmental Psychology, 42(1), 59-69. Maxwell, K. L., Field, C. C., & Clifford, R. M. (2006). Defining and measuring professional development in early childhood research. In M. Zaslow & I. Martinez-Beck (Eds.), Critical issues in early childhood professional development (pp. 21-48). Baltimore, MD: Paul H. Brookes Munoz, M. A., & Chang, F. C. (2007). The elusive relationship between teacher characteristics and student academic growth: A longitudinal multilevel model for change. Journal of Personnel Evaluation Education, 20, 147-164. National Association for the Education of Young Children. (2009). Where We Stand on Standards for Programs to Prepare Early Childhood Professionals. Washington D.C. Retrieved from http://www.naeyc.org/files/naeyc/file/positions/programStandards.pdf National Association for the Education of Young Children and National Council of Teachers of Mathematics. (2002). Position Statement, Early Childhood Mathematics: Promoting Good Beginnings. Washington, DC: Author National Research Council. (2001). Adding it up: Helping children learn mathematics. J. Kilpatrick, J. Swafford and B. Findell, B. (Eds.). Mathematics Learning Study Committee, Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academies Press. National Research Council. (2009). Mathematics learning in early childhood: Paths toward excellence and equity. Committee on Early Childhood Mathematics, Christopher T. Cross, Taniesha A. Woods, Heidi Schweingruber, Editors. Center for Education Division of Behavioral and Social Sciences and Education, Washington DC: The National Academies Press. Pepper, K. L., & Hunting, R. P. (1998). Preschoolers' counting and sharing. Journal for Research in Mathematics Education, 29(2), 164-183. Platas, L., Klein, A., & Starkey, P. (2006). How Do Two Years of Professional Development In a Preschool Math Curriculum Affect Teachers' Classroom Practices? Proceedings of Head Start's Eighth National Research Conference. Powell, D. R., & Diamond, K. E. (2012). Promoting early literacy and language development. In R. C. Pianta (Ed.), Handbook of early childhood education (pp. 194-216). New York, NY: Guilford Press. RAND. (2003). Mathematical proficiency for all students: Toward a strategic research and development program in mathematics education. RAND Mathematics Study Panel, Deborah Ball, Chair. Santa Monica, CA: Author. Romano, E., Babchishin, L., Pagani, L. S., & Kohen, D. (2010). School readiness and later achievement: Replication and extension using a nationwide Canadian Survey. Developmental Psychology, 46(5), 995-1007. Sabol, T. J., & Pianta, R. C. (2012). Recent trends in research on teacher-child relationships. Attachment and Human Development, 14(3), 213-231. Sarama, J., & Clements, D. H. (2009). Early childhood mathematics education research. New York, NY: Routledge. KNOWLEDGE OF MATHEMATICAL DEVELOPMENT SURVEY Sarama, J., DiBiase, A.-M., Clements, D. H., & Spitler, M. E. (2004). The professional development challenge in preschool mathematics. In D. H. Clements, J. Sarama & A. -M. DiBiase (Eds.), Engaging young children in mathematics: Standards for early childhood mathematics education (pp. 415-446). Mahwah, NJ: Erlbaum. Saxe, G. B., Guberman, S. R., & Gearhart, M. (1987). Social Processes in Early Number Development. Monographs of the Society for Research in Child Development, 52(2). Streiner, D. L., & Norman, G. R. (2008). Health measurement scales: A practical guide to their development and use. New York: Oxford University Press. Varol, F., Farran, D., Bilbrey, C., Vorhaus, E. A., & Hofer, K. G. (2012). Improving mathematics instruction for early childhood teachers: Professional development components that work. NHSA Dialog, 15(1), 24-40. Wiersma, W., & Jurs, S. G. (2005). Research methods in education: an introduction (8th ed.). Boston, MA: Allyn & Bacon. Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson III Tests of Cognitive Abilities. Itasca, IL: Riverside