Dialog, 23(2), Copyright © 2020, ISSN: 1930-9325 An Investigation into the Curricula (Type and Quality) Used by Early Childhood Educators Rachel E. Schachter University of Nebraska-Lincoln Shayne Piasta The Ohio State University Laura Justice The Ohio State University Most preschool-aged children spend time in Head Start and other center-based care. Thus, early educators’ use of a quality curriculum is essential to maximizing children’s learning. We examined the curricula utilized by educators working in diverse settings with children ages 3 to 5, focusing on key features of quality curriculum identified from the literature. Most educators (75%) reported using formal curricula. Creative Curriculum and HighScope were the most common; 6% of educators reported no curriculum use. There was a lack of consensus regarding what constitutes curriculum with 16% of participants reporting the use of materials generally not considered curriculum (e.g., learning standards). Although most educators were using a curriculum that included some key features, less than 15% were using curricula with evidence of effectiveness for supporting children’s learning. Findings have important implications for supporting practice and future research. Keywords: curriculum, early childhood, educators, preschool INTRODUCTION Curriculum is a critical component of classroom instruction and, in early childhood “provides a framework for planning an age-appropriate program” (Dodge, p. 1178). As such curriculum is the foundation upon which educators and programs create environments, physical and interactional, that support the growth of individual children. From a theory of change perspective (Schindler, McCoy, Fisher, & Shonkoff, 2019), the curriculum is a set of strategies employed by educators and programs that should result in children’s learning. The importance of curriculum in early RESEARCH ARTICLE 22 SCHACHTER ET AL. childhood is underscored by recommendations of professional organizations regarding what should constitute curricula (e.g., National Association for the Education of Young Children [NAEYC], 2003; National Center on Quality Teaching and Learning [NCQTL], 2015) and extensive research efforts to understand how curricula contribute to children’s learning (e.g., Preschool Curriculum Evaluation Research Consortium [PCER], 2008). Given this role of curriculum, many consider it as an important lever for improving the quality of instruction in early childhood education programs (Jenkins & Duncan, 2017; Weiland, McCormick, Mattera, Maier, & Morris, 2018). With the surge in attendance in early childhood programs and increased U.S. public funding supporting early childhood education, attention to curriculum and its use in early childhood programs is rising (Markowitz, Bassok, & Hamre, 2018). There are many state and federal policies requiring the use of curricula (e.g., U.S. Department of Health and Human Services [DHHS], 2010), and the majority of states include the use of curriculum as an indicator of quality in their Quality Rating and Improvement Systems (QRIS; Quality Compendium, 2019). Thus, external policy frameworks are playing a larger role in the uptake and implementation of early childhood curriculum. One common theme that emerges from many early childhood stakeholders, including state and federal constituencies, is that curricula should be chosen by educators and programs to meet the needs of their specific context, including children and families, and also be grounded in early childhood research. There are a variety of curricula options available for use in early-education programs serving 3- to 5-year-old children (NCQTL, 2015), yet little is actually known about what curricula are most widely used or the features of these curricula. This is important for understanding how curricula are supporting educators to implement high quality instruction. Thus, the purpose of the present study was to examine what early childhood educators reported using as classroom curricula and the features of these curricula. What is Curriculum in Early Childhood? Identifying a common definition of curriculum across the field of early childhood is challenging. For early childhood programs and educators, policy documents from funding or supervisory agencies are one resource for understanding what constitutes a curriculum. For example, the National Center on Quality Teaching and Learning (NCQTL) from the Office of Head Start defines a curriculum as a: …written plan that is based on sound child development principles, is consistent with program performance standards overall, and includes: Goals for children’s development and learning; Experiences through which children will achieve the goals; Roles for staff and parents to help children achieve these goals; and Materials needed to support the implementation of a curriculum. (p. 1, 2012) Other federal and state organizations such as the What Works Clearing House (WWC; US Department of Education, Institute of Educational Sciences) and state departments of education (SDE; including in the state in which data for this study were collected) identify similar definitions for curricula. These include the idea that curricula are written and focused on facilitating children’s CURRICULA USE 23 learning in a variety of content domains such as language and literacy, math, science, and social emotional development. Indeed, many QRIS requirements stipulate the use of written lesson plans that align with state learning standards across these domains (Quality Compendium, 2019). Also, often included in the definition of curriculum is a delineation of how such content is to be targeted via a scope (or range) of what that content should be and a specific sequence (or order) for teaching the content (e.g., Maker, 1986; Whitehurst, 2009). However, these definitions are by necessity left relatively broad in order to allow educators to implement curricula that is appropriate for their own teaching style and setting. Looking more generally at conceptualizations of early childhood curriculum, there is often an understanding that curriculum should be informed by theories about how children learn and develop (see Williams, 1999 for a historical review). The early childhood field typically tends to favor child-centered approaches that build from child interests to develop learning experiences across the cognitive, language, social-emotional, and physical domains (Jones, 2012; Jones & Nimmo, 1994; Kostelnik, Soderman, Whiren, & Rupiper, 2019; NAEYC, 2009). Accordingly, some have argued for the importance of emergent curricula, which develop based on the needs and the interests of specific children. This view is operationalized in different ways throughout the field and across curricula. In some cases this approach is integrated with research on learning, which underscores the importance of meeting children where they are and individualizing learning experiences to help children develop along a trajectory of skills (Burchinal, Peisner-Feinberg, Pianta, & Howes, 2002; Clements et al., 2011; Duncan et al., 2007). One way to meet children where they are developmentally is to use age-appropriate assessments, screening tools, or work sampling to inform instructional planning; curricula might include these to support the selection of learning activities. There are also approaches such as Montessori, Project Approach, or Reggio Emilia (Hall et al., 2010; Katz, Chard, & Kogan, 2014; Montessori, 2012) that embed a holistic, child-centered orientation within their overall frameworks that subsequently inform curricula development. These theoretical approaches are often listed as a type of curricula but differ slightly in that educators actually generate the curriculum based on these approaches in contrast to more manualized curricula. Finally, opportunities for learning through play are also often considered important in early childhood curricula (Golinkoff, Hirsh-Pasek, & Singer, 2006; Nicolopoulou, 2010; Wood, 2004); although some have suggested decreasing the focus on play in favor of engaging children in real-life activities (Lillard & Taggart, 2018). What Are Key Features of Quality Curricula? Although, to our knowledge, no existing framework identifies features of quality curricula used in early education settings, the available literature suggests a number of such key features. To identify these, we conducted an informal but comprehensive review of a variety of documents on early childhood curricula use including research studies, policy documents, and publications by early childhood organizations in order to identify features considered characteristic of quality curricula (see Table 1). Although not exhaustive, the review allowed us to identify key features with converging support through theory, best practice recommendations, and/or empirical research (noted in the table). Importantly, documents from professional organizations (e.g., NAEYC) and funding agencies (e.g., NCQTL) which were likely to be available and used by programs and educators to identify quality curricula were included. We identified these features as: including 24 SCHACHTER ET AL. learning goals, providing support for planning, specifying scope of the content, specifying sequence of the content, using assessment, and including mechanisms for culturally appropriate family involvement or outreach. Table 1 provides a definition of each feature and references to documents supporting its use as part of curriculum. It should be noted that across the literature, there is much variability and lack of specificity in the description of these features. Table 1 Key Features of Quality Curricula Feature Definition (coding criteria in italics) Rationale Includes Learning Objectives Curriculum has learning objectives for students. Categorized based on level of specificity of learning objectives from no learning objectives, broad, somewhat specified, specified. Celements & Sarama, 2007; Fantuzzo 2011*; Justice et al., 2010*; NAEYC, 2003; NCQTL, 2012; Skibbe et al., 2015 Support for Planning Curriculum has lesson plans that were tied to learning objectives. Curriculum must have had lesson plans to be coded in this category. Coded as yes or no. Bierman et al., 2008*; Fantuzzo et al., 2011*; NAEYC, 2003; NCQTL, 2012; Skibbe et al., 2015 Specified Scope Curriculum identifies the content that should be learned. Categorized based on the level of specificity about the content from no scope, broad, somewhat specified, to specified. Clements & Sarama, 2008*; Fantuzzo et al., 2011*; Justice et al., 2010*; Maker, 1986; NCQTL, 2012; Whitehurst, 2009 Specified Sequence Curriculum identifies the order in which content should be addressed. Categorized based on the level of specificity about the sequence from no sequence, broad, somewhat specified, to specified. Clements & Sarama, 2008*; Fantuzzo et al., 2011*; Justice et al, 2010; Maker, 1986; NAEYC, 2003; Whitehurst, 2009 Includes Assessment Curriculum contains a corresponding assessment to help understand how children are meeting curricular goals. Coded as yes or no. Clements & Sarama, 2008*; Confrey & Stohl, 2004; Fantuzzo, 2011*; NAEYC, 2003; NCQTL, 2012; Weiland & Yosikawa, 2013* Includes Training Curriculum provides training. Coded for whether there was professional development, supplementary materials, and/or additional materials. Chambers et al., 2010; Clements et al., 2011*; Domitrovich et al., 2009*; Markowitz et al., 2017; NAEYC, 2003; NCQTL, 2012; Fantuzzo et al., 2011*; Lonigan et al., 2015*; Yoon, et al., 2007; Family Involvement Curriculum has ways to include home and school connections. Fantuzzo et al., 2011*; NCQTL, 2003 Note. * indicates studies with child-level outcomes Additionally, another way to determine quality of curriculum is to identify how it is aligned with early childhood research (see Table 2). There are two ways to think about research as it connects with curricula – curricula may be research based or research tested (Duke & Martin, 2011). A CURRICULA USE 25 curriculum that is research based is grounded in the extant literature, such that it is aligned with research findings. Research tested, however, indicates that the curriculum itself has been studied for its impacts on children’s learning. Research-tested curricula have been receiving more attention over the past ten years through seminal studies such as the PCER report (2008) and greater focus on rigorous outcome research through WWC. As a result, there is an accumulating evidence-base regarding the efficacy of specific curricula. Notably, this body of work highlights the efficacy of content-specific curricula or what some researchers term “skill-focused” curriculum (e.g., Jenkins et al., 2019; Nguyen, 2017; Weiland & Yoshikawa, 2013) or “domain-specific” curriculum (Weiland, 2016). These curricula target one or two specific content areas or learning domains, in contrast to curricula that are designed to cover all of the learning domains holistically. There is evidence that these types of curricula are effective at improving the targeted outcomes. For example, although Jenkins et al. (2018) found an advantage for holistic curricula in improving the quality of classroom processes, only content-specific curricula led to positive effects on children’s school readiness outcomes. Additionally, in a recent meta-analysis, Nguyen (2017) found positive, small effects for skills-focused curricula and null effects for more holistically focused curricula on children’s academic skills. There is also evidence that integrating skills- focused and child-centered approaches may support academic skills. In a different meta-analysis, Chambers et al., (2016) found that approaches that incorporated skills-focused instruction with child-led activities had positive impacts on children’s language and literacy outcomes at the end of preschool and at kindergarten follow up. Table 2 Research Related Features of Quality Curricula Research – based Publisher states that the curriculum is supported by research. Coded as yes or no. Chambers et al., 2010; Clements & Sarama, 2007; Clements & Sarama, 2011*; Clements & Sarama, 2008*; Confrey & Stohl, 2004; Cross-Conn Powers 2014; Domitrovich et al., 2009*; Fantuzzo et al, 2011*; Justice et al., 2010*; NAEYC, 2003; Pence et al 2008* Content- specific Curriculum focuses on developing one content area and skills associated with that content area. Coded as yes or no. Jenkins et al., 2018; Lonigan et al., 2015*; Weiland et al., 2013* WWC Inclusion Curriculum has been examined in studies meeting the criteria for inclusion in What Works Clearinghouse. Coded as yes or no. WWC inclusion Evidence of Effectiveness Curriculum has been found to have positive effects either by WWC or other research studies. Coded as yes or no. See Appendix A Note. * indicates studies with child-level outcomes 26 SCHACHTER ET AL. What do We Know About the Curricula Used by Programs and Educators? Although the literature suggests various key features of quality curricula, we know much less about what curricula educators use and the extent to which these align with such features. It is important to note that curricula use may be somewhat determined by the requirements of funding agencies (e.g., Head Start or SDEs) or through a state QRIS (Connors & Morris, 2014). The expectation is that programs who receive these types of funding or ratings must use curricula that include features outlined by these agencies. It is unclear, however, if program selections always align with these recommendations. Furthermore, these requirements may not be exhaustive in listing features of quality curricula. There are a few studies that have examined curriculum use. Clifford et al. (2005) found that Creative Curriculum (Dodge et al., 2002) and HighScope (Epstein & Hohmann, 2012) were the most frequently used curricula in a sample of 240 educators in state-funded prekindergarten programs across four states. They reported that only 4% of educators did not use a curriculum and that this was more common in half-day programs. More recently, in a survey of 80 educators in one state, Cross and Conn-Powers (2014) found that only two educators reported using a curriculum that met WCC standards for evidence of effectiveness. However, the authors did not report on the features of the other curricula selected by educators thus leaving much to be learned about curricula use. Additionally, Jenkins and Duncan (2017) summarized a nationally representative sample of directors’ descriptions of selected curricula, finding that 32% of state- funded prekindergarten programs and 55% of Head Start programs used Creative Curriculum, and 7% of prekindergarten and 17% of Head Start programs used HighScope. Importantly, they found that 22% of prekindergarten directors and 5% of Head Start directors reported that they did not use a curriculum at all. In general, much of what the field knows about early childhood curricula use comes from the descriptive information about programs reported in research articles. Findings from these studies suggest that Creative Curriculum and HighScope tend to be the most frequently used curricula (Lonigan, Purpura, Wilson, Walker, & Clancy-Manchetti, 2013; Lonigan & Shanahan, 2010) and align with the research already described. None of these studies, however, provided an in-depth analysis of curricula use or how the curricula that educators used align with key features of quality. PURPOSE The purpose of the current study was to identify the curricula in use in a large number of early childhood classrooms across one state, representing a variety of early childhood settings including Head Start. We were particularly interested in the extent to which reported curricula aligned with the key features of quality curricula identified in the extant literature. To that end, we posed the following research questions: 1. What curricula are used by early educators in programs serving 3- to 5-year-old children across one midwestern state? 2. To what extent do the curricula used by early educators across the one state demonstrate key features of quality? CURRICULA USE 27 We used multiple research methods to address our questions. First data were collected via survey and summarized with descriptive statistics (RQ1) and then a content analyses of the identified curricula was conducted (RQ2). METHOD Participants Data for this study were collected as part of a larger project evaluating a state-sponsored professional development program. Participation was open to all early childhood educators in the state who served preschool-aged children (i.e., 3- to 5-years old) and volunteered to enroll in the free-of-cost professional development provided on an annual basis by the SDE (see Piasta et al., 2017 for a detailed description of recruitment and the professional development). A total of 546 educators participated in the larger project; of these, the 497 who completed a background questionnaire and answered items regarding curricula use were included in the present sample. Early childhood education in the state is similar to that of other states in that programs are funded by a variety of mechanisms – public (including Head Start), private, and hybrid. In addition to stipulating licensing requirements for programs, the state also has an established QRIS system that includes ratings dependent upon the use of developmentally appropriate screening tools and assessments as well as the use of a curriculum that meet state criteria (SDE, 2016). Almost all participants were female (95.57%) with an average of 11.36 years of early childhood teaching experience (SD = 7.93). Almost 40% reported that they were working in Head Start- affiliated classrooms; overall, most participants’ (74.04%) early childhood programs received some type of subsidized funding either through federal or state funding. Educators’ highest degree obtained included a high school diploma (15.29%), an Associate’s degree (22.13%), a Bachelor’s degree (31.19%), and a graduate degree (23.94%; 7.44% unreported). Educators were 80.08% White, 15.90% African American, 0.60% Asian, 0.20% Native Hawaiian or Pacific Islander, and 0.80% multiracial (2.41% unreported), with 1.21% reporting that they were Hispanic or Latinx. They were fairly equally distributed across urban (26.16%), suburban (30.58%), and rural (33.40%) locations (9.86% unreported). Procedures Educators completed a background questionnaire that included a variety of questions, including demographic information. To understand curricula use, the questionnaire included the item, “Please identify what types, if any, curricula you used in your classroom during the most recent school year and list by name.” We used all responses to this item to address our first research question. Each individual curriculum reported by educators was cataloged and then descriptive statistics were used to provide information regarding overall participant curricula use. Notably, educators were able to report the names of multiple curricula and 16.50% (n = 82) gave multiple responses. To address our second research question, we conducted a content analyses (Hsieh & Shannon, 28 SCHACHTER ET AL. 2005) to identify presence or absence of curriculum features identified in the extant literature. We began with list of all curricula that participants reported using (RQ1). We then screened the list to determine if what was identified by participants constituted a curriculum. For the purposes of this study, we used the WWC definition of curriculum identified in their protocol for reviewing early childhood programs as, “A curriculum is a set of activities, materials, and/or guidance for working with children in classrooms that has a clearly identified name, includes a write-up/description, and can be replicated by others based on written guidance, staff training, or technical assistance.” Educators’ responses that constituted general approaches/principles to child development, assessments, or educational materials other than a curriculum (e.g., state learning standards, “sensory-based approach”) were excluded from coding as these did not meet our definition of curricula. Additionally, some educators provided only a publisher (e.g., “Scholastic”), and we were therefore unable to determine the exact curriculum to code. Others indicated that the curriculum was “educator-created” and although educator- or school-created curricula have the potential to meet our definition of curriculum, we did not have follow up access to these curricula and thus were unable to code them and they were excluded as well. It is not our intention to place differential value on these types of curricula, rather we are unable to include them here as there were no documents for us to analyze. This left a pool of 35 unique curricula that met our definition and were used by participants for which we attempted to obtain additional information and code for curriculum features. We collected curriculum materials and related information (e.g., information provided online by developers or publishers) for 32 of these curricula; we were unable to obtain sufficient materials to code three identified curricula (i.e., Innovations, Catechesis of Good Shepherd, PreschoolFirst). We categorized these 32 curricula containing formal documentation as formal curricula and refer to them as such for the rest of the study. The content analyses was conducted based on a priori codes (Miles & Huberman, 1994) generated from the extant literature regarding quality curricula. These included general features of the curriculum (see next section and Table 3) as well as features of quality curriculum (see next section and Tables 1 and 2). A trained undergraduate research assistant used the collected curriculum materials and the a priori coding schema to code each curriculum. The first author double coded 25% of the curricula to establish interrater agreement. Interrater agreement was determined by dividing the total number of agreements between coders by the total number of agreements plus disagreements between coders at the individual code level. Agreement was high, ranging from 78% to 100% across the coding categories (M = 92%). General Features. To give a general overview of the curricula being used by early childhood educators, we coded each with respect to a number of basic, descriptive categories. These were features often described in definitions of curricula, but were not commonly identified by theory, best practice recommendations, and/or empirical research as key features of quality curricula. This included whether the curriculum was available for purchase, whether the curriculum was intended for preschool-aged children (age-appropriate), and whether the curriculum was described as based in early childhood or educational theory. For the latter, we also coded the specific theory or theorist referenced. Additionally, we coded the content areas covered in the curriculum, whether the curriculum provided lesson plans for educators to use, whether lessons plans were scripted (i.e., provided step-by-step instructions for what educators might say CURRICULA USE 29 or do), and whether the curriculum was emergent in that educators generated learning topics within the general framework of the curriculum. For example, to identify descriptive features of the Project Approach, the book Engaging Children’s Minds: The Project Approach (Katz, Char, & Kogan, 2014) was reviewed to identify that it is age-appropriate (p. 18), is theory based (pgs. 5- 15), covers a variety of domains and is emerging (pgs. 3-4), but does not have pre-specified lesson plans or templates. As state mandates are an important lever for encouraging educators to use curricula in early childhood, we were also interested in whether the reported curricula met state curricula selection guidelines. Thus, we coded whether each curriculum adhered to the first four of five guidelines for the state’s QRIS: “The curriculum a program utilizes must be written, research‐based, comprehensive, appropriate to the age group served, and show alignment to the program's identified assessment process” (SDE, 2016, p. 3). We were unable to code whether curricula were aligned with “the program’s identified assessment process” as we did not have this information for each early childhood program in which participating educators worked. Because of the absence of written lesson plans, the Project Approach was coded as not meeting state guidelines. Key Features of Quality Curricula. Based on our review of the extant literature, we generated a list of codes related to the key features of quality curricula. These codes along with descriptions and rationales are listed in Table 1. Specifically, we were interested in whether curricula contained key features of quality curricula and to what extent they were present in the curricula. For some of these features we coded them as present (Y) or absent (N) in the curricula (e.g., includes an assessment or lesson objectives). For other features (learning objectives, scope, and sequence), we coded for the level of specificity provided within the curriculum. This was to gauge how much information was provided to educators regarding implementing the curriculum. These were coded as none (N), broad (general list/information; B), somewhat specified (additional information, more detailed content; SS), or highly specified (very detailed list; HS) following the way that the SDE provided varying levels of detail to educators regarding the Early Learning Standards (2015). Appendix A provides definitions and examples of the specificity coding. To continue with the Project Approach example, a “no” code was given for lesson objectives, scope, sequence, and assessment because projects are generated based on children’s interests and the lesson plans and learning goals are specified while developing the project (pgs. 21-52). It was coded as “broad” in specifying learning objectives as four major goals are listed (p. 12). We also coded what type of research evidence there was to support the efficacy of the curriculum. This included examining the curricular materials to see if these stated that the curriculum was research based. We also coded for whether the curriculum was evidence based (research tested) in two ways. First, we searched for studies of the curricula meeting the WWC (U.S. Department of Education, Institute of Educational Sciences) criteria for credible and reliable evidence of effects on children’s outcomes. Second, we indicated if any additional studies (i.e., not included in the WWC) provided evidence of effects. The latter were identified by conducting a search of ERIC and PsycInfo as detailed in Appendix B. Combining these studies with those included in WWC, we were able to code for whether or not there was evidence that a curriculum improved outcomes for preschool-aged children. Both the WWC and the additional studies were coded as of the end of May 2016 as this was when the educators were actually using the curricula. Again, returning to the Project Approach, the authors reported that the curriculum is research based (pgs. 30 SCHACHTER ET AL. 5-15) aligning with research on children’s learning; however, there are no evidence of effects, either in WWC or in other studies. RESULTS What Do Early Childhood Educators Report Using as Curricula? We used descriptive statistics to address our first question regarding educators’ curriculum use. Table 3 summarizes educators’ reports of the curricula used in their classrooms and includes the number and percentage of educators who provided each response; note that the categories are not mutually exclusive as educators could provide multiple responses. Educators most frequently reported using Creative Curriculum (53.12%). Seven additional curricula were reported by at least five educators in the sample: HighScope (9.05%), Handwriting Without Tears/Get Set for School (2.62%), Montessori (2.21%), Let’s Begin with the Letter People (1.61%), The Core Knowledge Preschool Sequence (1.61%), Everyday Mathematics (1.61%), and Mother Goose Time (1.01%). Educators also reported using 27 other formal curricula, but each was used by four or fewer educators. Twenty-nine educators reported using an “educator- created” curriculum (5.84%), and 29 educators reported not using any curriculum in their classrooms (5.84%). In addition, 80 educators provided several other responses when asked to report the curricula that they used (16.10%). Some reported a publishing company but did not give enough information to identify a specific curriculum (Houghton Mifflin Harcourt-Saxon Early Learning, 3.02%; Scholastic, 0.40%; Pearson/Scott Foresman, 0.20%). Thirty-four educators reported the state early learning standards as their curriculum (6.84%), and one educator reported that their curriculum was “state developed.” Additionally, some educators reported an assessment system as their curriculum (AEPS, 5.84%; ATI Galileo, 0.60%). A few educators reported the “Head Start Outcomes Framework” or “Head Start” as their curriculum (0.60% and 0.20%, respectively). One educator reported a set of educational materials as curriculum (MC PreK Curriculum; 0.20%), and one educator reported a pedagogical approach as curriculum (Sensory-Based Approach; 0.20%). Although it is possible that educators did not understand what constitutes a curriculum, this is an important finding which we will return to later. Of the educators who responded with these “other” responses, 52 indicated these as their only curricula; thus, together with educators who reported “no curriculum,” we calculated that 81 educators did not indicate using any curriculum in their classrooms (16.30%). What Are the Features of Curricula Used by Early Childhood Educators? In order to address our second research question regarding the features of the curricula utilized by educators we used the findings from the content analyses in two ways. First each curriculum was defined individually (Table 3). Second, findings were enumerated (Dey, 2003) such that we could describe the overall features across curricula and by educator use. General Features. Table 3 includes general information for the formal curricula that were reported by educators and could be coded (n = 32 formal curricula). The vast majority were available for purchase (84.38%), and all but two were appropriate for preschool-aged children CURRICULA USE 31 Table 3 General Characteristics of Formal Curricula Educator use Curriculum descriptive characteristics Educator response Count Percent (n = 497) Available for purchase For PS Theory -based Theory Content Lesson plans Lessons scripted Emerging Meets state criteria Formal curriculum 384 77.26% The Creative Curriculum for Preschool 264 53.12% Y Y N LL, M, W, S, SS, CA N N Y Y The HighScope Preschool Curriculum 45 9.05% Y Y Y R LL, M, S, SS, SSk, CA N N Y Y Handwriting Without Tears/Get Set for School 13 2.62% Y Y N LL, W, M Y Y N N The Montessori Method 11 2.21% N Y Y M LL, W, M, GM, CA N N Y Y Let's Begin with the Letter People 8 1.61% Y Y N LL, M, S, SS, BM Y N N N CoreKnowledge Preschool Sequence 8 1.61% N Y N LL, W, M, S, BM, GM, CA Y N Y Y Everyday Mathematics 8 1.61% Y N N M Y N N N Mother Goose Time Preschool Curriculum 5 1.01% Y Y Y B, D, Em, G, H, P, R LL, W, M, S, SS, SE, GM, CA Y N N Y 32 SCHACHTER ET AL. Educator use Curriculum descriptive characteristics Educator response Count Percent (n = 497) Available for purchase For PS Theory -based Theory Content Lesson plans Lessons scripted Emerging Meets state criteria The Project Approach 4 0.80% Y Y Y RE LL, W, M, BM, SS, SSk, S, GM, CA N N Y N secondStep 4 0.80% Y Y N SE Y Y N N Conscious Discipline 4 0.80% Y Y N BM N N Y N Read, Play and Learn! 4 0.80% Y Y N N N Y N The DLM Early Childhood Express 3 0.60% Y Y N LL, M, S, SS, SE, GM, CA Y Y N Y A Beka Book Homeschool Curriculum 3 0.60% Y Y N R, LL, M, CA Y Y N Y Opening the World of Learning (OWL) 3 0.60% Y Y N LL, M, S, SS, SE, GM, CA Y N N Y Read It Again-PreK! 3 0.60% N Y N LL Y Y N N Reggio Emilia Approach/Experience 2 0.40% N Y Y RE LL, W, M, BM, SSk, GM, CA N N Y N Pinnacle 2 0.40% Y Y Y P, G, E R, LL, S, SSk, CA Y Y N Y Gee Whiz Education 2 0.40% Y Y Y P, Er, S, V LL, M, S, SS, SE, BM, GM, CA Y Y N Y CURRICULA USE 33 Educator use Curriculum descriptive characteristics Educator response Count Percent (n = 497) Available for purchase For PS Theory -based Theory Content Lesson plans Lessons scripted Emerging Meets state criteria Fountas & Pinnell 2 0.40% Y N Y FP LL N N Y N Little Treasures 2 0.40% Y Y N LL, W, M, S, SS, CA Y Y N Y The Emerging Language and Literacy Curriculum 2 0.40% Y Y N LL, W, M, SE, CA Y Y N Y Innovations: The Comprehensive Preschool Curriculuma 1 0.20% HighReach Learning Curriculum 1 0.20% Y Y N LL, M, S, SS, SE, GM, CA Y N Y Y Explorations with Young Children: A Curriculum Guide from the Bank Street College of Education 1 0.20% N Y Y F, W, J, D, P, Er, I, SM LL, M, S, GM, CA N N Y N WEE Learn 1 0.20% Y Y N R N N Y N How to Handle Hard- to-Handle Preschoolers 1 0.20% Y Y N Y Y N Y The PATHS Curriculum 1 0.20% Y Y N SE Y Y N N 1-2-3 Learn Curriculum 1 0.20% Y Y Y N LL, BM, SSk, GM, CA N N N N Happily Ever After for Prekindergarten - The 1 0.20% Y Y N LL Y Y N N 34 SCHACHTER ET AL. Educator use Curriculum descriptive characteristics Educator response Count Percent (n = 497) Available for purchase For PS Theory -based Theory Content Lesson plans Lessons scripted Emerging Meets state criteria Superkids Reading Program God Loves Me Storybooks 1 0.20% Y Y N R Y Y Y N Catechesis of Good Shepherda 1 0.20% PreschoolFirsta 1 0.20% S'Cool Moves Inc 1 0.20% Y Y N Y Y N N Anti-Bias Education 1 0.20% Y Y N SE N N Y N Educator-created 29 5.84% No curriculum 29 5.84% Other 80 16.10% State Learning Standards 34 6.84% Assessment, Evaluation, and Programming System Curriculum 29 5.84% Houghton Mifflin Harcourt-Saxon Early Learning 15 3.02% Head Start Outcomes Framework 3 0.60% ATI Galileo 3 0.60% Scholastic 2 0.40% Pearson/Scott Foresman 1 0.20% State Developed 1 0.20% CURRICULA USE 35 Educator use Curriculum descriptive characteristics Educator response Count Percent (n = 497) Available for purchase For PS Theory -based Theory Content Lesson plans Lessons scripted Emerging Meets state criteria Head Start 1 0.20% MC PreK Curriculum 1 0.20% Sensory-Based Approach 1 0.20% Note. PS = preschool, Y = yes, N = no, LL = language and literacy, W = writing, M = math, S = science, BM = behavior management, GM = gross motor, CA = creative arts, SS = social studies, SE = socioemotional development, SSk = social skills, R = religion, R = Rogoff, M = Montessorri, B = Bronfenbrenner, D = Dewey, Em = Emde, G = Gardner, H = Hotz, P = Piaget, RE = Reggio Emilia, Er = Erikson, S = Smilanksy, V = Vygotsky, FP = Fountas and Pinnell,, F = Freud, W = Heinz Werner, J = Harriet Johnson, I = Susan Isaacs, and SM = Lucy Sprague Mitchell. aUnable to obtain sufficient materials to code for these curricula. 36 SCHACHTER ET AL. Table 4 Key Features of Quality Curricula Formal curriculum Learning Obj. Lesson Obj. Scope Sequence Assess Training Family Involve Research based WWC Inclusion Evidence of Effects The Creative Curriculum (Preschool) SS N B N Y PD, R Y Y Y N The HighScope Preschool Curriculum B N SS N Y PD, R Y Y N PPE Handwriting Without Tears/Get Set for School SS Y HS HS Y PD, R Y Y N PPE Montessori Method N N N N N PD Y Y N PPE Let’s Begin with Letter People SS N SS SS N R N Y Y PPE CoreKnowledge Preschool Sequence HS Y SS SS Y R Y Y N N Everyday Mathematics SS Y HS SS N PD Y Y Y PPE Mother Goose Time Preschool Curriculum B Y B B Y R Y Y N N The Project Approach B N N N N R N Y N N SecondStep B Y HS HS Y R N Y N PPE Conscious Discipline SS N N N N PD, R Y Y N N Read, Play and Learn! SS N B SS N R N Y N N The DLM Early Childhood Express B Y HS HS Y R N Y Y PPE A Beka Book Homeschool Curriculum SS Y HS HS Y PD Y Y N N Opening the World of Learning (OWL) HS Y B B N R Y Y N PPE CURRICULA USE 37 Formal curriculum Learning Obj. Lesson Obj. Scope Sequence Assess Training Family Involve Research based WWC Inclusion Evidence of Effects Read It Again PreK! SS Y HS HS N PD N Y N PPE Reggio Emilia Approach/Experience N N N N N Y N N N Pinnacle B N HS HS N PD N Y N N Gee Whiz Education HS N HS HS N R Y Y N N Fountas & Pinnell SS N SS SS Y PD N Y N N Little Treasures B Y HS HS Y PD, R Y Y N N The Emerging Language and Literacy Curriculum B Y HS HS N N N Y Y N HighReach Learning Curriculum SS Y N N N R Y Y N N Explorations with Young Children: A Curriculum Guide from the Bank Street College of Education N N B N Y N Y N N N WEE Learn HS N B B N N Y N N N How to Handle Hard- to-Handle Preschoolers N N HS HS N PD, R N Y N N The PATHS Curriculum HS Y HS HS N PD Y Y N PPE 1-2-3 Learn Curriculum B N B SS N PD, R N Y N N Happily Ever After for Prekindergarten - The Superkids Reading Program B N HS HS N PD, R N N N N God Loves Me Storybooks N N HS HS N R Y N N N 38 SCHACHTER ET AL. Formal curriculum Learning Obj. Lesson Obj. Scope Sequence Assess Training Family Involve Research based WWC Inclusion Evidence of Effects S'Cool Moves Inc. B Y HS HS N PD, R N Y N N Anti-Bias Education HS N HS N N PD Y Y N N Note. Obj = objectives, WWC = What Works Clearinghouse, N = no/none, B = Broad, SS = somewhat specified, HS = highly specified, Y = yes, PD = professional development, R = other training resources, PPE = potentially positive effect(s) on at least one outcome CURRICULA USE 39 (93.75%). Approximately one-third were described as based in theory (31.25%), with the following theories/theorists represented: Bronfenbrenner, Dewey, Emde, Erikson, Fountas and Pinnell, Gardner, Hotz, Rogoff, Montessori, Piaget, Reggio Emilia, Smilansky, Freud, Werner, Johnson, Isaacs, Sprague Mitchell, and Vygotsky. The majority included language and literacy content (68.75%), with 59.38% including math content and 56.25% including creative arts content. In descending order, other content included: science (40.63%), gross motor development (34.38%), socioemotional development (31.25%), social studies (28.13%) and writing (25.00%), behavior management (21.88%), social skills (15.63%), and religion (12.50%). Most curricula provided lesson plans (62.50%), and many of these were scripted lessons (43.75%); 43.75% of curricula were considered emerging. Fourteen curricula met the state criteria for use in early childhood programs (43.75%). Key Quality Features. Additional features of these curricula, specifically the extent to which they exhibited key features of quality curricula, are presented in Table 4, and the number and percentage of educators using curricula exhibiting these features are presented in Table 5. Curricula ranged in specification of learning objectives (15.63% of curricula provided no learning objectives, 34.38% provided broad learning objectives, 31.25% provided somewhat specified objectives, and 18.75% provided highly specified objectives), but most educators reported using curricula with somewhat specified learning objectives (58.61%). Less than half of curricula (43.75%), used by only 10.29% of educators, provided lesson plans tied to learning objectives. Many curricula provided a highly specified scope (50.00%) or sequence (45.16%); however, most educators used curricula with broad scopes (52.52%) and no sequence (86.35%). Approximately one-third of curricula included integrated or aligned assessments (34.38%), with many educators using these curricula (70.38%). Most curricula included options for educator professional development (54.84%), either alone or supported by other resources, to support implementation, but 9.68% of curricula did not provide any implementation support; the curricula with no implementation support were infrequently used by educators. Most curricula also provided ways of fostering family involvement (59.38%), with a corresponding majority of educators using curricula exhibiting this feature (73.11%). Research-Related Features. Finally, with respect to research, all but five curricula were described as being supported by research (i.e., research based; 84.38%). Fewer curricula, however, have been reviewed by the WWC (15.63%) or tested in other empirical studies (additional 21.88%). Of those reviewed by the WWC, three (9.38%) showed potentially positive effects on at least one child outcome. All seven tested in other empirical studies showed potential positive effects, although not all of these studies were of high quality. Although many educators (58.61%) used curricula that had been reviewed by the WWC, few (14.71%) used curricula with evidence of potentially positive effects on child outcomes. 40 SCHACHTER ET AL. Table 5 Key Features of Quality Curricula by Educator Count Percent (n = 476a) Learning objectives Broad 60 12.61% Somewhat specified 279 58.61% Highly specified 16 3.36% Included lesson objectives 49 10.29% Scope Broad 250 52.52% Somewhat specified 60 12.61% Highly specified 42 8.82% Sequence Broad 7 1.47% Somewhat specified 22 4.62% Highly specified 36 7.56% Included assessment 335 70.38% Training Professional development 17 3.57% Resources 24 5.04% Professional development and resources 321 67.44% Family involvement 348 73.11% Research based 363 76.26% Reviewed by What Works Clearinghouse 279 58.61% Evidence of effects 70 14.71% Note. a Denominator does not include those educators who reported only using an educator-created curriculum or the three formal curricula that we were unable to code (i.e., Innovations, Catechesis of Good Shepherd, PreschoolFirst). DISCUSSION The purpose of the present study was to describe curricula use in a diverse sample of early childhood educators, including those working in Head Start settings, and identify how those curricula aligned with key features of quality curricula. We found that there are a variety of curricula available to and used by early childhood educators to support their classroom practice but few of these exhibited features of quality curricula. These findings suggest that the field continues to have difficulties defining what constitutes curricula in early childhood and highlight potential issues in terms of the quality of curricula that are currently being used by educators. This work has several implications for how we support educators in implementing curricula that provides high quality instruction. Before discussing our major findings, we note that using this sample was an important extension of the current literature. In addition to confirming existing literature focused on early childhood programs serving children in either Head Start or state-funded prekindergarten programs, the sample includes participants from a larger variety of preschool programs than have been CURRICULA USE 41 investigated previously. In general, our findings align with other research demonstrating that Creative Curriculum is one of the most frequently used curricula in early childhood classrooms, with over half of participants reporting use of this curriculum, followed by the other most commonly reported curriculum, HighScope (Clifford et al., 2005; Jenkins & Duncan, 2017; Lonigan et al., 2013). We also found similar rates as to the number of educators reporting no curriculum use at all, which represented about 6% of the sample. Importantly, by including a diverse sample of educators, we were able to extend the literature by examining key features of a variety of curricula not typically described in by researchers. This is critical, as funding mechanisms (including Head Start) typically do not mandate use of specific curricula; thus, a better understanding of the myriad curricula choices available is important for informing educators’ choices. Reaching a Clearer Conception of What Constitutes Curricula in Early Childhood It was challenging to identify curricula across educators’ reports in this study. Indeed, we found that over 15% of educators incorrectly identified what constituted a curriculum; conflating curriculum with other types of instructional supports such as assessments and learning standards. Although instructional supports such as curriculum, learning standards, and assessments should be used in an integrated and systematic way to inform practice in early childhood contexts (Kostelnik et al. 2019; NAEYC, 2009), these are three separate instructional supports. In part, this confusion may be due to the field not achieving consensus in defining or operationalizing what constitutes “curriculum,” as described previously. This can be observed in the many ways curricula are documented and made available for educators. Although organizations such as Head Start provide definitions of curricula, these definitions are intentionally left broad so that programs can make curricular decisions that are appropriate for their contexts. However, this can be problematic. We found great variety not only in the curricula available to and used by educators, but also the characteristics of and types of materials included in the curricula – indeed, there was quite a range of what was represented in the formal curricula reported. In some cases, curricula were highly specified with daily, sometimes moment-to-moment, educator practices. In contrast, some curricula were actually theoretical orientations to curriculum development (e.g., Project Approach, Montessori) that do not explicitly provide activities, scope, or sequence; rather, these guide educators’ generation of curricula based on individual children and the classroom context. These curricular approaches are often grouped into the category of curricula but do not embody the same features as more prescribed/manualized curricula. An additional complexity is in understanding features of school- or educator-created curricula. One limitation to this study is that we were unable to code these types of curricula which also have the potential to meet many of the quality features. It is, however, important to note that a limited number of participants (n = 29, 5.84%) reported using this type of curricula. That is not to say that one type of curriculum is to be favored over the other, rather we hope to highlight that there are many complexities in understanding differing types of curricula – something that may be particularly challenging for programs and educators as they seek guidance in selecting curricula. Indeed, this challenge may be seen in commonly accepted features of early childhood curricula. For example, many curricula described themselves as emergent, yet this varied in implementation (e.g., child-driven projects in the Project Approach versus child-selected 42 SCHACHTER ET AL. activities in HighScope). Additionally, even the idea of play and how it might be enacted is of debate in early childhood (Lillard & Taggart, 2018). Thus, more consideration for how the field understands early childhood curriculum and what constitutes quality is merited. This is particularly important given the growing policy mandates around curriculum that often delineate what should be incorporated into curriculum. Having a clearer conception across the field is critical if curriculum is to achieve its potential in supporting instruction that promotes children’s learning. Commonly Used Curricula were not Well-aligned with Quality Features When considering alignment with quality features of curricula, we found that many curricula fell short of meeting these criteria. Critically, only ten of the curricula examined in this study had evidence of positive effects on child outcomes (with only three that had WWC evidence), and these curricula were used by less than 15% of educators. In other words, most educators were using curricula that did not have evidence of effects on children’s outcomes. It is important to note that there are challenges to measuring the impacts of curricula on children’s outcomes (Burchinal, 2018; PCER, 2008), particularly for curricula that are theoretically driven, as proximal measures of change are hard to identify. Furthermore, evidence of effects are not the only quality feature to consider in curriculum use. However, as the field of early childhood continues to strive towards improving early childhood teaching practices to support high quality instruction, understanding the evidence-base regarding curricula is essential. Additionally, the lack of alignment between reported curricula and quality features suggests that the available and commonly used curricula may be under-supporting educators. In particular, we noted across the curricula that there was a range of specificity in scope and sequence of learning content, with less than 10% of educators using a curriculum with highly specified scopes and sequences. Moreover, most educators were using curricula that did not provide lesson plans (89%) or highly specified lesson objectives (96%). Thus, educators using these curricula needed to make the decisions about content and sequence and how to package those into a specific lesson plan. In some ways this format for curriculum development can be advantageous for expert educators who are able to appropriately develop and sequence activities to support children’s learning. They can create lessons and lesson objectives as they go, building on their knowledge of children’s development as well as their knowledge of the individual children in their classroom (Shulman, 1987). This, however, is a difficult task (Schachter, 2017) and requires a sophisticated knowledge of both child development and curriculum development which, given the range of backgrounds of early childhood educators (IOM & NRC, 2012), not all educators may have. Notably, more than half of participants used a curriculum that provided training – although the nature of this training and whether it attended to curriculum implementation fidelity is unknown as well as whether or not educators and programs utilized the training is unknown and merits further investigation. Thus, both in-service and preservice training may need to provide more information and support to educators in using these types of curricula to design and implement instruction (Markowitz et al., 2018). CURRICULA USE 43 Researchers’ Role in Moving the Field Forward Researchers have a critical role to play in supporting both the development of curricula and understanding the effectiveness of curricula. Importantly, there is limited research on the extent to which many of the curricula used by educators in this study impact children’s learning (see Tables 1 and 2). Only five curricula that educators in this study reported using have been investigated in studies meeting WWC standards for rigorous causal research. Researchers need to continue to focus on existing curricula, especially those in current use by educators, in order to provide better information to the field regarding the efficacy of what educators are currently using on a day-to-day basis. Additionally, more research is necessary to understand the characteristics of quality curricula and how these support learning and instruction. Research that examines both the how individual features are supportive of high-quality instruction and how these curricula are effective in specific contexts for specific children is critical (Schindler et al., 2019). For example, what specifically about scope and sequence are important in implementing curricula? Alternatively, how are family and school connections effectively fostered by curricula and does this differ across families/communities? Understanding the elements of these curricula that contribute to children’s learning may be particularly illuminating in the design and implementation of new curricula as well as the refinement and improvement of popular curricula. This could be accomplished both through implementation science research (Durlak, 2010; Mendive, Weiland, Yoshikawa, & Snow, 2015; Wasik & Hindman, 2014) and/or design research (Clements, 2007; Davis, Palinscar, Smith, Arias, & Kademian, 2017). Additionally, given the challenges in measuring impacts of curriculum on child outcomes (Burchinal, 2018; PCER, 2008), researchers need to continue to refine research tools and methods, as well as engage in more longitudinal research following children into the middle elementary grades in order to find evidence regarding the efficacy of a variety of curricula in supporting children’s long-term academic outcomes. Finally, more evidence is needed to identify the merits of holistic versus skills focused curriculum. Whereas the field has generally been oriented to holistic curricular approaches (Jones, 2012; Jones & Nimmo, 1994; Kostelnik et al., 2019; NAEYC, 2009; Williams, 1999), recently researchers have focused on the evidence linking skills-focused curricula to child outcomes. If researchers are going to continue in advocating for more skills-focused curricula use, then more research should also investigate the efficacy, usability, and feasibility of implementing multiple content-focused curricula in real-world contexts. Currently, most educators, both in this study and in other research (Clifford et al., 2005; Jenkins & Duncan, 2017; Lonigan et al., 2013) used holistic curricula to target multiple domains simultaneously (e.g., Creative Curriculum and HighScope). Use of these curricula may be pragmatic in addressing multiple goals and, logistically speaking, it may be more manageable to implement one curriculum given the complex work of teaching and managing children, families, and contextual variables (Cohen, Raudenbush, & Ball, 2003). Emerging evidence does seem to suggest that two content-specific curricula can be used in conjunction to achieve child outcomes (e.g., Lonigan et al., 2015; Weiland & Yoshikawa, 2013) and just over 15% of educators in this study reported using multiple curricula. How educators can successfully combine curricula in order to build an emergent curriculum that addresses all the domains deemed to be important for kindergarten readiness (Duncan et al., 2007; Magnuson, Meyers, Ruhm, Waldfogel; 2004) is an important consideration as the field continues this line of research 44 SCHACHTER ET AL. (Markowitz et al., 2018). Furthermore, how educators can do this without losing other important instructional elements such as higher quality classroom interactions (Jenkins et al., 2018) or opportunities for play (Golinkoff et al., 2006; Nicolopoulou, 2010; Wood, 2004) is necessary. As a whole, researchers need to do a better job of bridging the research-to-practice gap. This need is clear in the number of educators using nonevidence-based curricula and the variable alignment with quality curricula features. Research findings need to be better communicated to educators and practitioners as they seek to implement curriculum in their classrooms and programs. Although most of the reported curricula were research-based, only 14% of educators were using a curriculum that had evidence of effects from either a WWC review or another study. One problem is that the means for investigating whether or not curricula are evidence-based may not be readily accessible for educators or programs. For example, one current concern is that practitioners are not aware of the WWC as a resource or how to use it to inform their programmatic decision- making (Schneider, IES, December 2018). Other evidence regarding efficacy is often reported in research studies published in academic or professional outlets which may be difficult to access without licensing rights to specific journals. Thus, as researchers, we need to think more deeply about how to provide this type of information to early childhood educators and programs. LIMITATIONS AND CONCLUSION There are some limitations of this study. First, as mentioned previously, we were unable to obtain or code educator/school created curricula. More research is needed to understand how these curricula align with key curricular features. Furthermore, we have no data regarding the implementation of curricula, our data were limited to educator reports regarding the curricula that they use and thus cannot speak to implementation of the quality features. We recognize that there is much variability in curriculum implementation (e.g., Fantuzzo et al., 2011; Lonigan et al., 2015; PCER, 2008) and that this is just as, if not more, important than the curriculum itself for child outcomes. Future research will need to examine the enactment of key-features of curricula as they are connected with child outcomes within individual program settings. Overall, our findings illuminate the ongoing lack of consensus as to what constitutes quality curriculum in early childhood and indicate the need for more research and training to assist those working in early childhood programs. Specifically, educators need more support in understanding what constitute features of quality curricula. Researchers have an important role to play in moving the field forward both in studying the efficacy of existing curricula as well as how to implement features of quality curricula more generally. As a field, we need better mechanisms for describing curricula and identifying quality features. More work is needed if we are to bridge the research- to-practice gap and ensure that curricula are effectively supporting teachers in providing high quality learning opportunities that have positive impacts for young children. CURRICULA USE 45 ACKNOWLEDGMENTS The research reported here was supported by the Institute of Education Science, U.S. Department of Education, through Grants R305E100030 and R305B12008 to the Ohio State University. This work would not have been possible without the contribution of Ann O’Connell and individuals from the Early Childhood Quality Network including Dennis Sykes, Sharon Sullivan, Kathryn Rider, and Melissa Ross, along with the Ohio Department of Education’s Office of Early Learning and School Readiness. We also wish to thank the early childhood agencies, administrators, educators, PD facilitators and research staff members without whom this work would not have been possible. The opinions expressed are those of the authors and do not necessarily represent the views of the Institute of Education Sciences, U.S. Department of Education, Early Childhood Quality Network, or Ohio Department of Education. REFERENCES *indicates a reviewed curriculum *Abrams & Company. (2001). Let’s begin with the letter people®. Waterbury, CT: Author. *Anonymous. (2014). Destination reading. Boston, MA: Houghton Mifflin Harcourt. *Anonymous. (2015). Gee whiz fantastic friends. (n.p.): Gee Whiz Education. *Anonymous. (2011). HighReach learning. Greensboro, NC: Carson Dellosa Publishing Group. *Anonymous. (2014). Homeschool preschool lesson plans. Pensacola, FL: Abeka. *Anonymous. (2015). Mother goose time preschool curriculum. Grawn, MI: Experience Early Learning. *Anonymous. (2005). Pinnacle. Duluth, Georgia: ChildCare Education Institute. *Anonymous. (2014). PreschoolFirst. Reston, VA: The Source for Learning, Inc. *Appelbaum, M. (2013). How to handle hard-to-handle preschoolers: A guide for early childhood educators. New York, NY: Skyhorse Publishing, Inc. *Bailey, R. A. (2001). Conscious discipline. Oviedo, FL: Loving Guidance. *Bell, J., & Bell, M. (1995). First grade everyday mathematics: Teacher's manual and lesson guide (2nd ed.). Chicago, IL: Everyday Learning Corporation. Burchinal, M. (2018). Measuring early care and education quality. Child Development Perspectives, 12, 3-9. Burchinal, M. R., Peisner-Feinberg, E., Pianta, R., & Howes, C. (2002). Development of academic skills from preschool through second grade: Family and classroom predictors of developmental trajectories. Journal of School Psychology, 40, 415-436. Chambers, B., Cheung, A., & Slavin, R. E. (2016). Literacy and language outcomes of comprehensive and developmental-constructivist approaches to early childhood education: A systematic review. Early Childhood Research Quarterly, 18, 88-111. https://doi.org/10.1016/j.edurev.2016.03.003 *Clements, D. H., & Sarama, J. (2003). DLM early childhood express math resource guide. Columbus, Ohio: SRA/McGraw-Hill. Clements, D. H., Sarama, J., Spitler, M. E., Lange, A. A., & Wolfe C. B. (2011). Mathematics learned by young children in an intervention based on learning trajectories: A large-scale cluster randomized trial. Journal for Research in Mathematics Education, 42, 127-166. doi:10.5951/jresematheduc.42.2.0127 Clifford, R. M., Barbarin, O., Chang, F., Early, D., Bryant, D., Howes, C., . . . Pianta, R. (2005). What is pre- kindergarten? Characteristics of public pre-kindergarten programs. Applied Developmental Science, 9, 126- 143. Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resources, instruction, and research. Educational Evaluation and Policy Analysis, 25, 119-142. doi:10.3102/01623737025002119 *Committee for Children. (1997). Second step: A violence prevention curriculum, grades 1–3 (2nd ed.). Seattle, WA: Author. Connors, M. C., & Morris, P. A. (2014). Comparing state policy approaches to early care and education quality: A multidimensional assessment of quality rating and improvement systems and child care licensing regulations. Early Childhood Research Quarterly, 30, 266-279. 46 SCHACHTER ET AL. *Core Knowledge Foundation (2000). Core knowledge preschool sequence: Content and skill guidelines for preschool. Charlottesville, VA: Author. Cross, A. F., & Conn-Powers, M. (2014). Making the intentional decision to use an effective curriculum to promote children’s learning. Early Childhood Education Journal, 42, 361-366. doi:10.1007/s10643-013-0623-4 *Derman-Sparks, L., & Edwards, J. O. (2010). Anti bias education for young children and ourselves. Washington, DC: NAEYC. Dey, I. (2003). Qualitative data analysis: A user friendly guide for social scientists. New York, NY: Routledge. Dodge, D. T. (1995). The importance of curriculum in achieving quality child day care programs. Child Welfare, 74 (1171-1188). *Dodge, D. T., Colker, L. J., & Heroman, C. (2002). The creative curriculum for preschool. Washington, DC: Teaching Strategies. Duke, N. K., & Martin, N. M. (2011). 10 things every literacy educator should know about research. The Reading Teacher, 65, 9-22. doi:10.1598/RT.65.1.2 Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., ... & Sexton, H. (2007). School readiness and later achievement. Developmental Psychology, 43, 1428-1446. doi:10.1037/0012- 1649.43.6.1428. *Epstein, A. S., & Hohmann, M. (2012). The highscope preschool curriculum. Ypsilanti, MI: HighScope Press. Fantuzzo, J. W., Gadsen, V. L., & McDermott, P. A. (2011). An integrated curriculum to improve mathematics, language, and literacy for Head Start Children. American Educational Research Journal, 48, 763-793. doi: 10.3102/0002831210385446 Farley, K. S., Brock, M. E., & Winterbottom, C. (2018). Evidence-based practices: Providing guidance for early childhood practitioners. Journal of Research in Childhood Education, 32, 1-13. doi: 10.1080/02568543.2017.1387205 *Fountas, I. C., & Pinnell, G. S. (2009). Leveled literacy intervention. Portsmouth, NH: Heinemann. Golinkoff, R. M., Hirsh-Pasek, K., & Singer, D. G., (2006). Why play = learning: A challenge for parents and educators. In D. G. Singer, R. M. Golinkoff, & K. Hirsh-Pasek (Eds.) Play = Learning: How Play Motivates and Enhances Children’s Cognitive and Social-Emotional Growth (pgs. 3-12). New York, NY: Oxford University Press. *Hall, K., Horgan, M., Ridgeway, A., Murphey, R., Cunneen, M., & Cunningham, D. (2010). Loris Malaguzzi and the Reggio Emilia experience. New York, NY: Bloomsbury. Hsieh, H., & Shannon, S. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15, 1277–1288. Jenkins, J. M., & Duncan, G. J. (2017). Do prekindergarten curricula matter? Washington, DC: Brookings Institute. Retrieved from: https://www.brookings.edu/wp-content/uploads/2017/04/duke_prekstudy_final_4-4- 17_hires.pdf Jenkins, J. M., Duncan, G. J., Auger, A., Bitler, M., Domina, T., & Burchinal, M. (2018). Boosting school readiness: Should preschool teachers target skills or the whole child? Economics of Education Review, 65, 107-125. Jones, E. (2012). The emergence of emergent curriculum. Young Children, March, 66-68. Jones, E., & Nimmo, J. (1994). Emergent curriculum. Washington, DC: NAEYC. *Justice, L. M., & McGinty, A. S. (2009). Read it again-preK! A preschool curriculum supplement to promote language and literacy foundations. Columbus, OH: Author. Justice, L. M., McGinty, A. S., Cabell, S. Q., Kilday, C. R., Knighton, K., & Huffman, G. (2010). Language and literacy curriculum supplement for preschoolers who are academically at risk: a feasibility study." Language, Speech & Hearing Services in Schools 41, 161-178. doi: 10.1044/0161-1461(2009/08-0058). *Katz, L. G., Chard, S. C., & Kogan, Y. (2014). Engaging children’s minds: The project approach. Denver, Colorado: Praeger. Kostelnik, M. J., Soderman, A. K., Whiren, A., & Rupiper, M. L. (2019). Developmentally appropriate curriculum: Best practices in early childhood, 7th Edition. Pearson. *Kusche, C. A., & Greenberg, M. T. (1994). The PATHS curriculum. Seattle, WA: Developmental Research and Programs. *Lang, J. (n.d.). 1-2-3 curriculum. Saint Paul, Minnesota: Pam Ball Designs. Lillard, A. S., & Taggart, J. (2018). Pretend play and fantasy: What if Montessori was right? Child Development Perspectives, 13, 85-90. doi: 10.1111/cdep.12314 *Linder, T. W. (1999). Read, play, and learn! Storybook activities for young children. The transdisciplinary play- based curriculum. Baltimore, MD: Brookes Publishing Company. CURRICULA USE 47 Lonigan, C. J., Phillips, B. M., Clancey, J. L., Landry, S. H., Swank, P. R., Assel, M., … Barnes, M. (2015). Impacts of a comprehensive school readiness curriculum for preschool children at risk for educational difficulties. Child Development, 86, 1773-1793. doi: 10.1111/cdev.12460 Lonigan, C. J., Purpura, D. J., Wilson, S. B., Walker, P. M., & Clancy-Manchetti, J. (2013). Evaluating the components of an emergent literacy intervention for preschool children at risk for reading difficulties. Journal of Experimental Child Psychology, 114, 111-130. Lonigan, C. J., & Shanahan, T., (2010). Developing early literacy skills: Things we know we know and things we know we don’t know. Educational Researcher, 39, 340-346. http://www.jstor.org/stable/27764607 Magnuson, K. A., Meyers, M. K., Ruhm, C. J., & Waldfogel, J. (2004). Inequality in preschool education and school readiness. American Educational Research Journal, 41, 115-157. doi:10.3102/00028312041001115 Maker, C. J. (1986). Developing scope and sequence in curriculum. Gifted Child Quarterly, 30, 151–158. https://doi.org/10.1177/001698628603000402 Markowitz, A. J., Bassok, D., & Hamre, B. (2018). Leveraging developmental insights to improve early childhood education. Child Development Perspectives, 12, 87-92. https://doi.org/10.1111/cdep.12266 Miles, M. B., Huberman, A. M., Huberman, M. A., & Huberman, M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed). Thousand Oaks, CA: Sage. *Mitchell, A., & David, J. (Eds.). (1992). Explorations with young children: A curriculum guide from the Bank Street College of Education. Lewisville, NC: Gryphon House, Inc. *Montessori, M., (2012). The Montessori method. United States of America: Renaissance Classics. *Morris, B., Morrow, A., & Yarbrough, B. (1996). WEE Learn Curriculum Guide for Three-Year-Olds. Nashville, TN: Lifeway Church Resources. National Association for the Education of Young Children. (2003). Early childhood curriculum, assessment, and program evaluation: Building an effective, accountable system in programs for children birth through age 8. Retrieved from: https://www.naeyc.org/sites/default/files/globally- shared/downloads/PDFs/resources/position-statements/pscape.pdf National Center on Quality Teaching and Learning. (2012, June). Choosing a preschool curriculum (Version 1.1). Retrieved from: https://cms.azed.gov/home/GetDocumentFile?id=585c5081aadebe14288f254c National Center on Quality Teaching and Learning. (2015). Preschool Curriculum Consumer Report. Author. Retrieved from: https://eclkc.ohs.acf.hhs.gov/sites/default/files/pdf/curriculum-consumer-report.pdf *Nederveld, P. (2015). Count the Stars! The Story of God’s Promise to Abraham and Sarah. Grand Rapids, MI: Faith Alive Christian Resources. Nicolopoulou, A. (2010). The alarming disappearance of play from early childhood education. Human Development, 53, 1-4. doi: 10.1159/000268135 *Olsen, J., & Knapton, E. (2008). Handwriting without tears (3rd ed.). Cabin John, MD: Western Psychological Services. *Ornes, J., Patterson, S., McMillan, D., Thomas, J. & Trumbower, E. (2017). The emerging language and literacy curriculum, (2nd ed.). Indiana, PA: Dynamic Resources. Piasta, S. B., Justice, L. M., O’Connell, A., Mauck, S., Weber-Mayrer, M., Schachter, R. E.… Spear, C. F. (2017). Effectiveness of large-scale, state-sponsored language and literacy professional development on early childhood educator outcomes. Journal of Research on Educational Effectiveness, 10(2), 354-378. https://doi.org/10.1080/19345747.2016.1270378 Preschool Curriculum Evaluation Research Consortium (2008). Effects of preschool curriculum programs on school readiness (NCER 2008-2009). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Washington, DC: U.S. Government Printing Office. Quality Compendium (2019). Build Initiative Strong Foundations for Our Youngest Children. https://qualitycompendium.org/top-ten/types-of-indicators *Rowland, P.T. (2003). Happily ever after for prekindergarten: The superkids reading program. Columbus, OH: Zaner-Bloser, Inc. Schachter, R. E. (2017). Early childhood teachers’ pedagogical reasoning about how children learn during language and literacy instruction. International Journal of Early Childhood, 49, 95- 111.https://doi.org/10.1007/s13158-017-0179-3 *Schickedanz, J.A., & Dickinson, D.K. with Charlotte-Mecklenburg Schools. (2005). Opening the world of learning: A comprehensive early literacy program. Parsippany, NJ: Pearson Early Learning. Schneider, M. (2018, December 17). A more systematic approach to replicating research. Institute for Educational Sciences [Director’s Remarks]. Retrieved from https://ies.ed.gov/director/remarks/12-17-2018.asp 48 SCHACHTER ET AL. Schindler, H. S., McCoy, D. C., Fisher, P. A., & Shonkoff, J. P. (2019). A historical look at theories of change in early childhood education research. Early Childhood Research Quarterly, 48, 146-154. doi: 10.1016/j.ecresq.2019.03.004 *Shanahan, T., Bear, D., Fisher, D., Gibson, V., & Rogers, F. (2007). Pre-K little treasures. Desoto, TX: MacMillan/McGraw-Hill. Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1– 22. doi:10.17763/haer.57.1.j463w79r56455411 U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, What Works Clearinghouse. U.S. Department of Health and Human Services (2010). The Head Start Child Development and Early Learning Framework: Promoting Positive Outcomes in Early Childhood Programs Serving Children 3-5 Years old. Arlington, VA: Head Start Resource Center. Weiland, C. (2016). Launching preschool 2.0: A road map to high-quality public programs at scale. Behavior Science & Policy, 2, 37-46. Weiland, C., McCormick, M., Mattera, S., Maier, M., & Morris, P. (2018). Preschool curricula and professional development features for getting to high-quality implementation at scale: A comparative review across five trials. AERA Open, 4, 1-16. doi: 10.1177/2332858418757735 Weiland, C., & Yoshikawa, H. (2013). Impacts of a prekindergarten program on children's mathematics, language, literacy, executive function, and emotional skills. Child Development, 84, 2112-2130. https://doi.org/10.1111/cdev.12099 Whitehurst, G. (2009). Don’t forget curriculum. Brown Center Letters on Education. Washington, DC: Brookings. Williams, L. R. (1999). Determining the early childhood curriculum: The evolution of goals and strategies through consonance and controversy. In C. Seefeldt (Ed.), The Early Childhood Curriculum: Current Findings in Theory and Practice (1-26). New York, NY: Teachers College Press. *Wilson, D.E., & Heiniger-White, M.C. (2014). S'cool moves for learning: Enhance learning through self-regulation activities. Camas, WA: S’cool Moves, Inc. Wood, E. (2004). Developing a pedagogy of play. In A. Anning, J. Cullen, & M. Fleer (Eds.), Early childhood education: society and culture (pp. 19–30). London: SAGE Publications Ltd. CURRICULA USE 49 Appendix A: Definitions and examples of specification categories for scope, sequence, and learning outcomes Scope Sequence Learning Outcomes Definition Scope refers to what should be taught or learned as part of the curriculum (Maker, 1986). Does the curriculum delineate the content to be covered and the level of detail provided about the content covers? Sequence refers to the how the content (scope) should be organized throughout the curriculum, the order of that delivery (Maker, 1986). If there is no scope, then there is no sequence as the two are tied together in the creation of curriculum. Learning outcomes state what the children will learn with specific reference to children’s learning. The categories reflect the level of specificity provided in the State Early Learning Standards. None Only lists the addressed content areas (e.g., language and literacy, math) without providing more specified detail about the content to be taught. If there is no scope or if there is not information provided about the order in which to address the content of the curriculum. The curriculum lists no learning outcomes or objectives for children. Broad Provides a general list beyond that of the overarching content area (e.g., phonological awareness) but does move beyond this level of detail. Provides a general order of content (e.g., address phonological awareness and letter knowledge at the same time). The curriculum broadly lists learning objectives outcomes for children but these do not move beyond basic content areas or general learning goals (e.g., “children will learn emergent writing skills” or “children will develop phonological awareness skills”). Somewhat- specified Provides additional information about content (e.g., teach rhyming and blending skills). Provides additional information about sequence (e.g., teach rhyme before blending). Provides more level of detail such as learning trajectories with milestones, or provides more detail about learning objectives beyond general skills (will be able to rhyme words, will be able to segment words). Highly- specified Gives a very detailed list of all of the content to be covered (rhyme “at” family). Gives very detailed listing of content sequencing (e.g., teach one-syllable rhymes before multi-syllable or “at” family before “og” family). Lists individual learning goals that are very detailed and micro-level components of specific skills. This could include specific learning objectives on lesson plans (learn the “og family of words”) 50 SCHACHTER ET AL. Appendix B: Search protocol and procedures for additional studies demonstrating evidence of efficacy for child outcomes The purpose of this category was to capture other studies of curricula that demonstrated evidence of efficacy for changing children’s outcomes not included in WWC. These were identified through a search of ERIC & PSYCHINFO with the name of the curriculum AND the following terms connected with OR: “group design,” “control group,” “treatment group,” “experiment*,” “intervention,” “pretests,” “posttests,” “experimental groups,” “matched groups,” “quasi- experiment*,” “efficacy,” “effectiveness,” “experiment*,” “randomized,” “control trial,” “random assignment,” “randomly assigned,” “regression discontinuity,” “single case,” “single subject.” Only studies that were peer-reviewed and written in English were included. When then reviewed the abstract and only included to determine if the study was of the curriculum and not a study that included the curriculum (i.e., the study must be of the curriculum with or without another curriculum). Furthermore, to be included studies had to examine preschool aged (3-5 years old) children's outcomes. Anything that had already been included via WWC (U.S. Department of Education, Institute for Educational Sciences) was excluded. Studies using the curriculum as a control condition were also excluded. This resulted in seven studies meeting the inclusion criteria. These were coded as to whether or not (Y/N) there was evidence of efficacy for children’s outcomes.