106 Global Education Review 5(3) ________________________________________________________________________________________________________________________ Global Education Review is a publication of The School of Education at Mercy College, New York. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC by 4.0), permitting all use, distribution, and reproduction in any medium, provided the original work is properly cited, a link to the license is provided, and you indicate if changes were made. Citation: Sitabkhan, Yasmin, Platas, Linda M. & Ketterlin-Geller, Leanne R. (2018). Capturing children’s mathematical knowledge: An assessment framework. Global Education Review, 5 (3), 106-124. Capturing Children’s Mathematical Knowledge: An Assessment Framework Yasmin Sitabkhan RTI International Linda M. Platas San Francisco State University Leanne R. Ketterlin-Geller Southern Methodist University Abstract This paper explores an innovative assessment framework for measuring children’s formal and informal mathematical knowledge. Many existing standardized measures, such as the Early Grade Mathematics Assessment, measure children’s performance in early primary grade skills that have been identified by researchers and policy makers as foundational and predictive of later academic achievement (Platas, Ketterlin-Geller, & Sitabkhan, 2016; RTI International, 2014). However, these standardized assessments only provide information on children’s mathematical ability as it pertains to skills and concepts that are a focus of school instruction, referred to as formal mathematics. While valuable, they leave unmeasured the mathematics that children use and develop as part of their everyday life, such as the strategies they use to solve simple arithmetical problems that arise as they move through their day (Khan, 1999; Saxe, 1991; Taylor, 2009). In this article, we draw from mixed methods studies which focus on capturing the informal mathematical skills that children develop outside of school in various contexts (Guberman, 1996; Nasir, 2000; Sitabkhan, 2009; Sitabkhan, 2015). We describe how the use of observations of children’s mathematical activities in natural settings and in subsequent cognitive interviews using mathematical tasks derived from those observations, can illuminate mathematical knowledge and skills that may otherwise remain hidden. We found that an assessment framework that focuses on both standardized measures of formal mathematical learning and contextualized measures of children’s everyday mathematics can provide a more complete and nuanced picture of children’s knowledge, and taken together can inform the development of curricular materials and teacher training focused on early learning. Keywords early grade mathematics, primary school, assessment Introduction Mathematical skills and knowledge have been useful to societies throughout history (Radford, 1997). Their importance has been recently _______________________________ Corresponding Author: Yasmin Sitabkhan, Senior Early Childhood Researcher International Education, RTI International, 701 13th Street, N.W., Suite 750, Washington, DC 20005 Email: ysitabkhan@rti.org Capturing children’s mathematical knowledge 107 highlighted by large-scale studies investigating the predictive power of mathematical skills on later academic achievement and economic well- being (Hanushek & Woessman, 2008; Watts et al., 2014). Beginning in 1995, children’s math knowledge has been measured at the population-level through easy-to-administer written tests beginning in the fourth grade (International Association for the Evaluation of Educational Achievement, 2011). However, these fourth grade and later skills are greatly influenced by earlier skills that are less amenable to written assessment. In addition, investigations into the effects of early mathematical development show a strong influence on children’s academic achievement across domains, including literacy (Claessens & Engel, 2013; Duncan et al., 2007). Because of the magnitude of these findings and the implications at both individual and societal levels, measuring early math skills has become a priority for governments in low and middle- income countries (MELQO Core Team, Technical Advisory Groups, & Steering Committee, 2017; Raikes et al., 2017). Ideally, such measurement should provide actionable information about children’s mathematical development to ministries of education. A variety of early grade assessments have been used to measure children’s mathematical development in low- and middle-income countries. Some of these assessments have been part of a citizen-led effort to ensure that children are receiving an adequate education and include the Annual Status of Education Report in India (ASER, 2016) and Uwezo in East Africa (2015a, 2015b). Others have been efforts by countries connected by a common language (i.e., Francophone countries in Africa) to better understand how their children are faring in developing mathematical skills and understandings (i.e., Programme for the Analysis of Education Systems [PASEC], 2015). Yet other assessments have been used by governments and non-governmental agencies in low- and middle-income countries to measure baseline mathematical competencies in their student population and to evaluate the outcomes from an array of interventions (Early Grades Mathematics Assessment [EGMA], RTI International, 2014). These existing assessments focus on the universal skills that researchers and practitioners agree form the foundation of early mathematical skills needed to learn more sophisticated mathematics. Left largely unmeasured by these existing assessments are the everyday, often referred to as informal, mathematical knowledge and skills that children develop as they solve problems that arise in their everyday life. As children play, run errands, help around the house, and participate in community events, problems arise that require mathematical solutions. For example, a child may need to distribute the evening bread evenly among three siblings or go to the market to purchase tomatoes for a meal. As they solve these problems, they develop and employ everyday mathematical skills that are distinct from the formal skills they learn in school (Khan, 1999; Saxe, 1991; Taylor, 2009). Everyday mathematical knowledge is a key piece of the puzzle when assessing young children’s mathematics. In this paper, we present an inclusive mathematics assessment framework that measures both formal and everyday mathematical knowledge and skills in low- and middle-income countries. This framework builds upon existing methodology for capturing children’s informal mathematics, along with assessments of formal mathematics, such as the EGMA and ASERs. By assessing both formal and informal mathematics, governments can obtain 108 Global Education Review 5(3) a comprehensive picture of the mathematics that their children know and do not know, and inform development of curricular materials that build from student’s prior knowledge and teacher training on effective instruction for early grades mathematics. A key innovation in this inclusive mathematics assessment framework is the introduction of tasks that measure children’s everyday mathematical knowledge. These tasks can be developed using a systematic and iterative process. First, through multiple, small- scale qualitative studies that take place in diverse cultural, linguistic, and geographic areas within a country, observations can be used to capture the types of mathematical problems that children solve during the course of their day. Second, tasks that mirror these everyday problems can be developed and subsequently administered to the same children who were observed. This data can be examined to ensure they capture both the everyday problems that arise for children, and their solutions to these problems. Finally, the tasks may then be combined with an assessment that measures formal mathematics for a representative sub- population of students within the country. These tasks will illuminate children’s everyday mathematical skills, skills that would otherwise remain hidden. This strategy provides information on children’s performance in both formal and everyday mathematics for the chosen populations. We begin our paper with a discussion of differences between formal and everyday mathematics. We follow with descriptions of some of the more widely used population-level formal mathematics assessments as well as reviewing existing methodology for capturing everyday mathematics. Finally, we detail our framework for measuring both formal and everyday mathematics at population-level. Formal and Everyday Mathematics Formal mathematics generally refers to symbolic, abstract mathematics. It is often called “school mathematics,” as it is the math that children learn in school (Nunes, Schliemann & Carraher, 1993). Everyday mathematics generally refers to the math that children use outside of school, such as the calculations they make when purchasing items in a market, or counting they may do while playing games. Everyday math is also called “out-of-school mathematics,” “street mathematics,” and “informal mathematics” (Nunes, Schliemann & Carraher, 1993). In groundbreaking work, Nunes, Schliemann, and Carraher (1993) investigated the differences between “street math” and “school math” among a population of schooled and unschooled participants in Brazil. Through observations and interviews with children and adults they found several heuristics that guide our understanding of these differences. Formal mathematics is often written, whereas informal math is more likely to be oral. Formal math emphasizes the movement from concrete to abstract, and aims to create generalizations of mathematical concepts. Informal math is concrete, and tied to objects and referents in the physical world. In order for children to develop a strong foundation in early mathematics concepts, and be prepared for increasingly complicated content, it is important to build conceptual bridges between formal and everyday math. Vygotsky’s (1986) framework of scientific and spontaneous concepts can be used here to better understand the relationship between formal and informal math. Formal mathematics is considered a scientific concept because it must be learned through the help of a more knowledgeable other. Formal math includes the Capturing children’s mathematical knowledge 109 ability to generalize and abstract a concept. For example, a child may learn that the “+” symbol means “add.” In contrast, everyday mathematics is considered a spontaneous concept because the knowledge and skills are acquired through interaction with the world around us and are local and specific to a given context. For example, a child may put objects together during play and know that by counting them all, she can produce the total amount. When scientific (formal) and spontaneous (informal) concepts converge, children begin to develop a deeper and more comprehensive understanding of mathematical concepts. The informal idea of putting objects together provides meaning to the word (add) and the addition symbol (+). The symbol (+) allows the child to apply that understanding not just to the objects in front of her but to other objects, as well as symbols. The interdependency between formal and everyday mathematics points to the importance of assessing both forms of knowledge. Figure 1 illustrates the differences between these two essential facets of early mathematical knowledge and skills, as well as why assessing both is important. Figure 1. Differences between formal and everyday mathematics 110 Global Education Review 5(3) As seen in Figure 1 above, a comprehensive analysis of students’ mathematical knowledge and skills paves the way for the development of curricular materials and teacher training on developing effective instruction that bridges informal and formal mathematics. Prior studies have been conducted on how to both assess children’s everyday mathematics and then to use this knowledge to develop instruction that connects formal and informal mathematics in the classroom (Brenner, 1998; Gonzalez, Andrade, Civil, & Moll, 2001). Brenner (1998) conducted a study investigating how children used money in their everyday lives to purchase items in a store after school, and how their use of mathematics in the store was connected (or disconnected) from the mathematics they were learning in the classroom. She found that almost all calculations children made in the store involved change- making; children rarely totaled the prices, and they often bought items that were close to one dollar. In contrast, classroom instruction on use of money focused on choosing items to buy and adding them together to calculate the total price. This disconnect caused children to initially develop two separate systems of arithmetic – the one they used outside of school and the one they were learning in school – and children did not make connections between the two. However, Brenner found that the teacher was able to bridge this divide by discussing with the students what they were doing outside of school, and how it was similar and different to what they were learning in class. Brenner’s study supports the idea that identifying the everyday mathematics that children use and connecting it to the formal mathematics in school can be a powerful pedagogical technique. However, there are limitations to generalizing or scaling-up this study. The measurement of the everyday mathematical knowledge and skills children were using was done by a highly experienced researcher, working with one classroom of children over the course of a school year. The pedagogical techniques developed jointly by the researcher and an experienced teacher are subsequently neither sustainable nor scalable. Another study aimed to support teachers in identifying students’ everyday mathematics knowledge. The Funds of Knowledge for Teaching program trained teachers to conduct mini ethnographic studies of their students to better understand the types of mathematics they were using outside of school, and plan instruction accordingly (Moll, Amanti, Neff, & González, 1992; Gonzalez et al., 2001). However, given the intensity of the training needed, supporting teachers to do this is neither scalable nor sustainable. In addition, teacher quality can be low in low- and middle-income country contexts, with many teachers receiving little to no training in early mathematics instruction (Akyeampong, Lussier, Pryor, & Westbrook, 2013). Mathematics instruction in pre-primary and early primary in these contexts is often composed solely of memorizing procedures. Given this, teachers do not have the requisite support, training, or time required to assess their students’ everyday mathematics and then plan instruction that bridges the two on their own. In such cases, community, district, or national-level governmental agencies can support this process. Many governments in low- and middle-income countries are already measuring formal mathematics at population- level using assessments such as the EGMA, ASER, or similar tools. We propose that by implementing an inclusive mathematics assessment framework that integrates tasks of Capturing children’s mathematical knowledge 111 everyday mathematics with these existing assessments of formal mathematics, governmental agencies can obtain a comprehensive picture of students’ mathematics. These data can then be used to develop curricular materials, such as supplemental problem sets that are tied to children’s everyday mathematics, and teacher training that provides teachers with tools to connect scientific and spontaneous concepts. These uses would not be possible without an inclusive assessment framework that accounts for both formal and informal mathematics knowledge and skills. How then, can governments best assess both formal and everyday mathematics for young children at population-level? To begin, we review current formal mathematics assessments to better understand how assessments were developed and how they are currently used. We then discuss an existing methodology for everyday mathematics. Population-level Assessments of Formal Mathematics For formal mathematics, we detail the EGMA, ASER, and Uwezo assessments. We first explain how the assessments were developed. We then provide an example of the type of data that comes from the assessments and discuss policy implications. In reviewing these assessments and their uses, it is important to attend to the potential limitations in accurately assessing all children. Notably, country-level reports on the results of these assessments highlight the existence of a considerable number of children who are unable to provide answers on any of these tasks, consequently receiving a total score of zero. There are policy and practice implications of labeling groups of children as “unable to do mathematics,” while in fact they possess important everyday mathematical knowledge, which could be captured by the proposed framework. As noted previously, the EGMA is administered to children in early primary grades to measure their formal knowledge and skills in early mathematics concepts. Results from the EGMA are used to inform policy makers, practitioners, and researchers about the effectiveness of existing educational policies, curricular programs or reforms, and instructional interventions supporting student achievement in mathematics (Platas, Ketterlin- Geller, & Sitabkhan, 2016). Content assessed on the EGMA focuses on early formal mathematics concepts that are predictive of future performance, including basic counting, magnitude comparisons, and simple operations. Counting skills are assessed by students’ ability to identify number names and complete number patterns by supplying the missing number. Magnitude comparison is assessed by asking students to discriminate between two numbers. Students’ ability to perform simple operations is assessed through fluency-based single-digit additional and subtraction, two-digit addition and subtraction, and word problems. Tasks were designed to sample key mathematical concepts at appropriate developmental levels for students in early primary grades. Where applicable, items are sequenced from least difficult to most difficult to allow students with emerging or advanced skills to demonstrate their knowledge and skills. Items that are embedded in a context such as the word problems are adapted to be culturally appropriate to the region and language of administration. During this process, people familiar with the customs and culture (e.g., ministry officials, teacher educators, teachers) provide recommendations for the appropriate context for the items. For example, in a context 112 Global Education Review 5(3) in which children ride buses to get to school, a word problem might be phrased as “There are four children in the bus. One child gets out of the bus. How many children are left in the bus?” In a different context in which children take boats to school, the same problem might be phrased as “There are four children in the boat. One child gets out of the boat. How many children are left in the boat?” The problem structure is retained but the contextual information is adapted to increase the relevance of the context. To illustrate the varied uses of the EGMA results, we highlight some key findings from research activities and policy initiatives. In Kenya, the EGMA was used as a dependent measure to evaluate the outcomes of an intervention program. Analysis of the EGMA results showed that the intervention program, The Primary Mathematics and Reading Initiative (PRIMR), improved the mathematics teaching of inexperienced and untrained teachers and, subsequently, children’s mathematical performance. This success convinced the Kenyan government to implement the PRIMR program country-wide (Piper, Ralaingita, Akach, & King, 2016). In Ghana, EGMA results were used to evaluate the quality of instruction by examining the difference in performance on conceptual tasks (i.e., two-digit subtraction) when compared to more procedural tasks (i.e., single digit addition). Findings pointed to poorer performance on conceptual tasks than procedural tasks; recommendations included a greater emphasis on conceptual teaching in mathematics in primary classrooms (Kochetkova & Brombacher, 2014). The Annual Status of Education Report (ASER) was created in India in 2005, to ascertain which literacy and mathematical skills children were learning in school. It is a nationwide survey conducted by citizen’s groups, and began as a literacy-only assessment in 1996. The mathematics section, added in 2005, includes number recognition, two-digit subtraction and three-digit by one digit division. Like many other countries, India (UNESCO, 2015) has moved beyond just counting the number of children enrolled in primary school, and turned their efforts to measuring outcomes. ASER is an annual nation- wide survey of over 700,000 children in 15,000 villages (ASER Centre, 2016a). It is the only annual source of information on children’s progress in mathematics in India. Because it is administered by citizens, it can be used as a tool in advocacy efforts to improve government provided education. In particular, it provides a tool for measurement in rural areas, where other forms of population-level assessment are limited. A variety of studies have used ASER to evaluate interventions and inform policy. From 2005 to 2014, and in 2016, the survey was used in all rural districts in India to measure children’s learning in primary school. These results are reported annually and are open- source. The Exploring Post Primary Schooling used the ASER to examine the skills of children in upper primary and found that 10% of children assessed could not solve subtraction problems. These outcomes and analyses inform curriculum development, academic resource planning and teacher education and training (ASER Centre, 2016b). Similar to the ASER, and implemented since 2009, Uwezo uses a citizen-implemented assessment to inform stakeholders about children’s competencies from age 6-16 years old in Kenya, Tanzania and Uganda. Concerns about the difference between enrollment in primary education and child education outcomes drove the effort. Also like ASER, Uwezo measures mathematics and literacy. The mathematics portion includes assessments of numeral and set Capturing children’s mathematical knowledge 113 matching, number recognition, number comparison, two-digit addition and subtraction, one digit multiplication, and division with dividends ≤20 (Uwezo, 2015a, 2015b). Recommendations from these reports in Uganda and Kenya include changes in education policies such as enforcing entry age, supporting early childhood education in an effort to reduce grade repeating, broader textbook distribution, strengthening schooling in the most challenging contexts, and active monitoring of learning outcomes. Similar to India, results showed that 5% of children in Kenya and 2% of children in Uganda aged 7-13 years old and in primary grades 3-7 respectively, could not perform even the easiest of the mathematical tasks in the assessment. Over the last 20 years, the Francophone Ministerial Conference for Education has provided expertise through its Programme for the Analysis of Education Systems (PASEC) to more than 20 countries in Sub-Saharan Africa, the Indian Ocean, the Middle East and South- East Asia. This has resulted in almost 40 national assessments. The mathematics portion of the assessment of children in grades 2 through 6 measures children’s skills in rote counting, numeral recognition, object counting, missing number series, number ordering, addition, subtraction, multiplication, measurement (perimeter, area, conversion and time), word problems, shape naming, spatial location vocabulary, size ordering, decimals and fractions. Children are assessed in primary schools, resulting in a representative sample of the school populations. Test administrators are trained, supervised and monitored by national teams (PASEC, 2015). Findings from a report on ten Sub- Saharan African countries indicate that overall, 16.2% of children have difficulty with even the easiest of the mathematics tasks. Recommendations in the report call for consideration of alternatives to grade repetition, increases in teacher quality, and increasing pre- primary attendance (PASEC, 2015). Across all of these instruments and uses, the recommendations are broad strokes across educational policies in teacher education, systems, curriculum, and resource allocation. Often, the results point to what children cannot do, and the skills that they have not yet acquired. However, missing from these results is the mathematics that children can do, and how they do it. For example, recall that Uwezo results from Kenya and Uganda found that 2% and 7% respectively, of children in primary grades 3-7 could not perform even the simplest tasks, such as addition and subtraction. These children most likely solve problems in their everyday lives that require addition and subtraction, whether in a game, household chores, the market, or other context. However, these children were unable to solve any tasks on the Uwezo. These children may not have recognized that the formal addition problems they were asked to solve, which were written with numerals and symbols (e.g. +, =) were the same problems they solve in their everyday life. Therefore, the conclusion was drawn that these children could not perform basic mathematics, when in fact the conclusion should have been that they cannot perform formal, school mathematics. More often than not, these children are among the most marginalized and attend poor-quality schools. They most likely can solve everyday math problems, and may have unique strategies with which to do so. To capture this knowledge, a different methodology is needed. Below, we describe one such methodology. 114 Global Education Review 5(3) Assessment of Everyday Mathematics There is an advantage to assessing everyday mathematics. Often, children from ethnic or language minorities, or certain regions, or genders, are categorized as “not knowing mathematics or not capable of learning mathematics.” As noted earlier, these children may be represented by very low or zero scores on formal mathematics assessments such as the EGMA, ASER, PASEC, or Uwezo assessments. The deficit approach which focuses on what skills children lack or cannot do prevails when discussing the most disadvantaged children, and many formal mathematics assessments contribute to this narrative. An assessment of everyday mathematics pushes back against this narrative, by revealing what children can do. Instead of focusing on lack of skills, we can appreciate and learn from the unique strategies that children have developed as they solve everyday problems. Ultimately it is the responsibility of the assessor to not reach conclusions about a child as “not knowing mathematics,” but instead to persevere to reveal the mathematics that this child does know. To capture the mathematics the child does know, observations and performance on mathematical tasks tied to the observations are needed. Saxe (1991), in a study aiming to understand the forms of mathematical problem solving of young street vendors in Brazil, developed an approach which used observations to understand the structure of the mathematics that children used during an observed event, and then children’s performance on mathematical tasks that mirrored the observations, to systematically understand the mathematics that children used. This method has been used by multiple researchers (Guberman, 1996; Khan, 1999; Nasir, 2000; Sitabkhan, 2009; Sitabkhan, 2015). In some cases, researchers then added similar tasks that measured formal mathematics knowledge (Saxe 1991; Taylor, 2009) or more complicated mathematics (Sitabkhan, 2009) as a comparison point. In a study in Mumbai, India, Sitabkhan (2009) used this technique to characterize the types of mathematical problem-solving that young children developed while selling small items on trains. The first phase of the study involved on-site observations of children. Observations were conducted over the course of 5 days, noting the age of the child, gender, and details of the transaction conducted while selling. After this, five of the vendors were shadowed for periods of 1-4 hours, where the researcher noted all transactions that occurred. Observations revealed the mathematics that children engaged in. First, there were three common pricing structures: (1) either a single item for a single price (e.g., one comb for 10 rupees), (2) multiple items for multiple prices (e.g., combs for 10 rupees and nail polish for 20 rupees), or (3) multiple items for ratio prices (3 hairbands for 10 rupees). All prices were either 5, 10, or 20 rupees, which were aligned with common denominations of currency. Children approached customers, displayed the goods they were selling, named the price, and then gauged the interest of the customer. If a customer was interested, the seller would calculate the total cost of the items for the customer, then count the money given to them and provide change if needed. From these observations, Sitabkhan developed tasks that aimed to mirror the mathematics used in transactions in order to more systematically understand the types of everyday mathematics strategies and problem- solving that children used while selling. Table 1 shows sample tasks that were created. Both simple and more difficult tasks were created in order to capture the mathematics observed. 115 Global Education Review 5(3) Table 1. Sample tasks created by Sitabkhan (2009) Simple More difficult Single Price 1. Let’s say you are selling pencils for 5 rupees each. A customer wants to buy 4. How much will it be? How do you know? 2. Now, a customer is having a party and wants to give pencils to all the guests. She wants to buy 56. Remember, the pencils cost 5 rupees each. How much will it be? How do you know? Multiple Prices 3. Let’s say that you are selling pencils for 5 rupees each and pens for 10 rupees each. A customer wants 4 pencils and 3 pens. How much will it be? How do you know? 4. Now, a customer comes up to you and says that she needs lots of new pencils and pens for her school. She wants 34 pencils and 40 pens. How much will it be? How do you know? Ratio Prices 5. Now let’s say that you want to sell these yellow pencils. You sell them for 5 rupees for 2. A customer wants to buy 6. How much will it be? How do you know? 6. Remember, you are selling 2 yellow pencils for 5 rupees. A customer again wants to buy many for her school. This time she wants 43. How much will it be? How do you know? The researcher asked these sellers (n=10) to solve these tasks in an individually administered assessment. A key feature of the tasks was to record how the seller solved the problem, and ask them to explain their answer. This type of probing provided insight into the strategies that the seller had developed to solve these problems. Table 2 show one seller’s solution to Task 2. 116 Global Education Review 5(3) Table 2. Child’s strategy for solving task related to observations. (Adapted from Sitabkhan, 2009) Task 2-more difficult: A customer is having a party and wants to give pencils to all the guests. She wants to buy 56. Remember, the pencils cost 5 rupees each. How much will it be? How do you know? Strategy Interpretation Participant #14 says: If 5 rupees for 1 pencil: then 50 rupees for 10 pencils 100 rupees for 20 pencils 150 rupees for 30 pencils 200 rupees for 40 pencils 250 rupees for 50 pencils then 30 rupees for 6 pencils 250 and 30 is 280 rupees Child begins by creating ratio of 50 rupees for 10 pencils, then appears to increase the unit (pencils) by 10 until reaching goal of 50 pencils. He then creates ratio of 30 rupees for the 6 pencils needed to get to 56 pencils in all, then adds 250 and 30 to arrive at final answer of 280 rupees. This seller used ratio prices by systematically increasing the unit (pencils) by 10 until reaching 50 pencils. The seller then calculated the cost for 6 pencils, and added to get to the total cost. This solution was oral. This is a different strategy than one sees in a school setting where students often use an algorithm to solve the problem on paper (Sitabkhan, 2009). Further analyses revealed commonalities in the strategies that sellers used to solve problems, including oral solutions and the use of common values (5, 10, and 20 rupees) used as operators to solve increasingly complex problems. The results of this study illustrate how the two-pronged approach of observations followed by designing tasks that mirror the observations can help illuminate mathematical strategies that may otherwise remain hidden on assessments of formal mathematics. Whereas assessments of formal mathematics aim to assess children’s understandings of universally agreed upon foundational skills, the process of assessing everyday mathematics focuses on capturing the everyday strategies that children develop to solve mathematical problems they face. A key question that arises when considering the design of everyday mathematics tasks is why do word problems not serve this function? As stated above, word problems are common in many formal assessments, such as the EGMA, and they are often adapted to fit the context. How, then do the word problems on formal assessments differ from those that were given to the young vendors in India? The origin of these problems differ. The everyday math problems were derived from observation and reflected the context and structure of the practice. In assessments such as the EGMA, the problems are based on research that has identified several different structures of word problems in early mathematics and corresponding student strategies to solve these problems (Carpenter, Fennema, & Franke, 1996). The structure of the problems of formal assessments, therefore, are dictated by the research. Table 3 below shows the structure of problems used on the EGMA. 117 Global Education Review 5(3) Table 3. Structure of word problems on the EGMA. (adapted from the EGMA Toolkit; RTI International, 2014) Problem Type Example Change: Result Unknown Two children are on the bus, three more children get on. How many children are on the bus altogether? Combine: Result Unknown There are six children on the bus, two are boys. The rest are girls. How many girls are there on the bus? Compare: Change Unknown There are two children on John’s bus and seven children on Mary’s bus. How many children must join John’s bus so that it has the same number of children as Mary’s bus? Change: Start Unknown Five children get on the bus. Now there are 12 children on the bus. How many children were on the bus to begin with. Sharing Four children share twelve candies equally between themselves. How many candies does each child get? Multiplicative There are five seats on the bus. There are two children on each seat. How many children are on the bus altogether? In comparison, in the study of everyday mathematics in India, the structure of the problem is derived from the observations. Although there may be some overlap with the problem structures of traditional assessments, the link to everyday practice is key. Children may approach these problems differently, using the strategies they developed within the constraints of the practice. For example, consider the ratio strategy that the child vendor in India used to solve Task 2 (seen in Table 1). Because this child was interacting with customers when solving mathematical problems, it was important to make sure the customer saw what they were doing, in order to avoid overpayment. The constant reference to the item in the child’s strategy may have served a purpose beyond just the calculation- it may have been a way to ensure the customer that they were being charged an accurate amount. This strategy may not have been apparent had the child been given a word problem where the mathematical calculation was the same (56 x 5) but the context different (e.g., There are five boxes with 56 pencils in each one. How many pencils are there in total?) Taylor (2011) discussed the importance of authenticity in mathematical tasks in the context of professional development that supports teachers in taking advantage of children’s out of school knowledge. Although a different context, the framework he used to help teachers evaluate tasks is useful. The framework had two dimensions: authenticity and connectedness to mathematics. For authenticity, tasks had to be based on real life tasks in which students were engaged. For connectedness to the mathematics, the math had to be naturally-embedded in the task. Taylor (2011) described connectedness as “concerned with the degree to which mathematics is a requirement for children to participate in a given practice” (p.13). 118 Global Education Review 5(3) Returning to the assessments, let’s apply this framework to a word problem from the EGMA: There are 6 children on the bus. Two are boys. The others are girls. How many girls are there on the bus? This problem is set in a familiar context for many children around the world, who may take buses to get around their city or town. For children in a more rural setting, where there are no buses, “bus” would be changed to “matatu,” or “tuk’tuk,” or “boat,” with the problem remaining the same. This is an example of an adaptation of a word problem to make a context more familiar to the child, thus creating an authentic problem in the eyes of the child. The problem structure remains a “combine result unknown.” The purpose of this task is to see if children can solve a “combine result unknown” problem- essentially, to measure their progress on applying their knowledge of mathematics to solving a problem that is seen as a core skill in the early years. By changing buses to “tuk’tuk” or some other means of transportation, the context is made familiar to children to support them in accessing the problem. However, as an everyday math problem, the question becomes, is this connected to the mathematics children do in their everyday life? That is, do children get on a bus, and have to calculate how many boy and girls are on the bus in order to participate in the bus ride? Is this a problem that children would need to solve during their bus ride? If the answer is yes, then it could belong on an everyday math assessment, as it is both authentic and connected to the mathematics. However, the use of buses by children in a country does not necessarily mean that calculating space on a bus is required in their everyday life. So, if the answer is no, then this problem does not give us a sense of children’s everyday mathematics, and a different methodology is needed to ensure that problems are authentic. How can we capture everyday mathematics in a way that is scalable and sustainable for governments of low- and middle- income countries? The above study in Mumbai provides insights only on this particular group of children’s everyday mathematical skills. At larger scale, it would be difficult, if not impossible, to capture all the different types of everyday mathematics. We turn to a hypothetical experiment below which articulates our framework for capturing both formal and informal mathematics at a larger scale. An Inclusive Assessment Framework for Assessing Formal and Everyday Mathematics Let’s imagine Country Y, a low-income country that is interested in revising/developing curriculum and curricular materials, and teacher training in early mathematics in order to improve the quality of teaching and ultimately improve student learning outcomes. Before embarking on this task, officials in Country Y would like to better understand existing levels of early mathematics to inform their revisions. Specifically, the country would like to know: 1. How are children in their country progressing in learning basic, foundational mathematical skills? 2. What other everyday mathematics do children use and know? By answering both of these questions, they hope to have enough information to develop curriculum and curricular materials, and teacher training. The Country officials decide to use the Capturing children’s mathematical knowledge 119 EGMA to answer the first question: how are children in their country progressing in learning the basic, foundational mathematical skills that have been identified in the research literature as important to future success. A plan for population sampling is constructed. An adaptation workshop is held with primary stakeholders, including teachers, principals, district education officers, nonprofit workers in the field of education, experts in the development of early mathematics curricula, assessments, and education policies, ministry of education representatives, and experts in the languages in which the EGMA will be administered. This workshop would provide information on the purpose of, and background information on the EGMA. The result of the workshop would be a successfully piloted EGMA consisting of the Core EGMA and any additional EGMA modules adapted to the languages and/or dialects and culture of the Country. This workshop would be followed by assessor/enumerator training on the EGMA and any other instruments to be included in the data collection. Prospective assessors/enumerators would be evaluated on their rapport with children and reliability in the EGMA administration. For the second question regarding everyday mathematics, Country Y would use the methodology detailed above, consisting of observations and tasks that mirror the observed use of mathematics. However, the main challenge they would face would be how to get information on everyday mathematics that comes from observations and analysis of performance on these tasks, an essentially qualitative method, at a scale large enough that the results can be generalized. This challenge is not trivial. On the one hand, the methodology of observations and assessment on tasks that mirror the observations ensures that the mathematics being captured is authentic, and connected to the mathematics that children actually use in their everyday life. As previous work has shown, children may use and develop different strategies and understandings based on the activity they are engaged in, as the children vending on trains did in India. On the other hand, if governments want to be able to understand and utilize the types of everyday mathematics that children are using, and given that it is not possible to visit every village in the country to conduct this study, there must be compromises made to be able to capture some type of everyday knowledge at scale. Given these challenges, can Country Y find a middle ground, that is true to the methodology while at the same time able to provide information that is useful to them in designing and/or revising curriculum and curricular materials? One possibility, detailed below, is for Country Y to pick sites for a qualitative study to develop and pilot tasks. These tasks could then be administered at the same time as an assessment of formal mathematics (e.g., EGMA), given to a sub- sample of students within the sampling framework. The sample could be chosen so that there is representation for factors that the government deems important, such as mother tongue, socioeconomic status, ethnic groups, gender, and/or region. First, the assessment team in Country Y would have to decide how to create their sample, and which factors to consider as important. If Country Y decided to divide by geographic area (if the country had a distinct geography that the team believed would affect the outcomes in mathematics, that would be a consideration). Country Y could decide to conduct two 120 Global Education Review 5(3) qualitative studies in each geographic area, if there were reasonable similarities between the geographic areas in terms of language spoken and industry. However, as there are differences between urban and rural, they could decide to conduct one study in an urban area in each region, and one in a rural area. They would contract with talented local researchers to engage in this work. After local researchers ( e.g., two per region) were trained, they would use an observation form to choose a common practice in which children engage where everyday math can be observed (e.g., buying at the market, buying treats/lunch before or after school, playing a game on the playground, interacting with caregivers at home). They then would observe children as they engaged in this practice and record the types of interactions children have with mathematics. Let’s zoom into Researcher W, who might be observing children in an urban Hill Region of Country Y. After spending 2-3 days in one community, Researcher W might see that children tend to be sent to the market after school to buy vegetables for their family. Researcher W would go to the market, and record any transactions where children between the ages of 5-8 buy vegetables. After several days of observing and recording, Researcher W might notice that almost all observations involve using ratio prices to calculate the total amount they need to give the vegetable seller. For example, Child G might be buying tomatoes that are 5 for 20 coins. Based on these observations, Researcher W would create several tasks that mirror and extend the observations. For example, one task might be “You are in the market. 5 tomatoes cost 20 coins. How much would it be for 10 tomatoes?” Another task might be “You are in the market. Five tomatoes cost 20 coins. How much would it be for 3 tomatoes?” Researcher W would then administer this task to a subsample of students between the ages of 5-8 in the community school to pretest the items, and ensure that they are capturing the types of mathematical knowledge and skills that were observed. These tasks would be added to the EGMA for a sub-sample of children. For each region, only the items from the qualitative study in that region, differentiated by urban/rural, would be added, to ensure that everyday mathematics test items are authentic and specific to local situations. The everyday mathematics tasks would be administered in addition to the EGMA by a subset of assessors across the different regions. These assessors would receive special training in how to administer the everyday math items, as they require assessors to notice how children solve problems and to be able to ask short follow-up questions, and then record the information accurately. When the data are collected and analyzed, Country Y would have answers to the two questions above. Armed with this information, the curriculum developers in Country Y could develop teacher guides, supplemental problems for use in the classroom, and guidance for district/regional educational offices, that aim to support children’s foundational math skills as well as making connections with everyday mathematics. For example, when introducing multiplication, guidance materials could use the examples from the market that Researcher W documented, and suggest supplemental problems that teachers might use. In addition, the results of the assessments would be shared with teachers during their annual teacher training. Teachers would be made aware of the types of mathematics that Capturing children’s mathematical knowledge 121 their students already know and use outside of school, and they would be provided with the means to integrate everyday mathematics into their teaching. By sharing the results and focusing not just on what a child lacks, but also what he or she already know, teachers can build upon prior knowledge. In addition, this knowledge may support teachers to see students as capable of learning and doing mathematics. Conclusion This inclusive framework for assessing formal and everyday mathematics highlights two important points. First, it is crucial for all children to learn and understand the foundational mathematical skills taught in early primary school. Second, children everywhere, regardless of schooling, are already using mathematics to solve problems and developing unique strategies to solve these problems. This knowledge may not be captured by extant measures of formal mathematical knowledge and skills. The inclusive mathematics assessment framework proposed in this manuscript brings these two sources of information together, with the ultimate aim of gaining evidence on children’s informal knowledge in the service of supporting them in learning formal mathematics. By pairing a methodology using observations to develop authentic tasks with existing population level assessments in the early grades, governments can be equipped with the right information to inform improvements in the quality of teaching and learning, and boost student learning outcomes. The results of an assessment using the inclusive framework we have proposed could be used in two ways. First, results can inform the development/revision of curriculum and curricular materials. For example, supplemental problem sets can be created that aim to connect children’s everyday mathematical knowledge with the formal mathematical knowledge that is being taught. If the assessment results reveal particular strategies that children use in out of school concepts, these strategies can be integrated into curriculum manuals for teachers that support them in lesson planning. Second, the results can inform teacher trainings, and provide teachers with simple strategies to bridge everyday and formal mathematics in the classroom. There is promise in the use of these results by local/regional/district level education officers to develop specific guidance for teachers. It may be that in some countries, one local district officer can be a point person for identifying and connecting local knowledge with school knowledge, not just for mathematics but other content areas as well. In other countries, this might happen at a community level, or even at the school-level. The addition of measurement of everyday informal mathematics can provide this information to parties regardless of their level. References ASER Centre. (2016a) 1996-2005: Foundations. Retrieved from http://www.asercentre.org/NGO/assessm ent/learning/education/outcomes/primar y/reading/p/133.html?p=158 ASER Centre. (2016b). Middle schools in India: Access and Quality: Endline report. Retrieved from http://img.asercentre.org/docs/Research %20and%20Assessments/Past/Education /exploringpostprimaryschooling.pdf Akyeampong, K., Lussier, K., Pryor, J., & Westbrook, J. (2013). Improving teaching and learning of basic maths and reading in http://img.asercentre.org/docs/Research%20and%20Assessments/Past/Education/exploringpostprimaryschooling.pdf http://img.asercentre.org/docs/Research%20and%20Assessments/Past/Education/exploringpostprimaryschooling.pdf http://img.asercentre.org/docs/Research%20and%20Assessments/Past/Education/exploringpostprimaryschooling.pdf 122 Global Education Review 5(3) Africa: Does teacher preparation count? International Journal of Educational Development, 33(3), 272-282 Brenner, M.E. (1998). Meaning and Money. Educational Studies in Mathematics, 36(2), 123-155. Carpenter, T., Fennema, E. & Franke, M. (1996). Cognitively guided instruction: A knowledge base for reform in primary mathematics. The Elementary School Journal, 97(1), 3-20. Claessens, A., & Engel, M. (2013). How important is where you start? Early mathematics knowledge and later school success. Teachers College Record, 115 (Retrieved from https://www.tcrecord.org/content.asp?co ntentid=16980). Duncan, G. J., Claessens, A., Huston, A. C., Pagani, L. S., Engel, M., Sexton, H., . . . Japel, C. (2007). School readiness and later achievement. Developmental Psychology, 43(6), 1428-1446. doi:10.1037/0012 1649.43.6.1428 Gonzalez, N., Andrade, R., Civil, M., & Moll, L. (2001). Bridging funds of distributed knowledge: Creating zones of practices in mathematics. Journal of Education for Students Placed at Risk Guberman, S. (1996). The development of everyday mathematics in Brazilian children with limited formal education. Child Development 67(4), 1609-1623 Hanushek, E. A., & Woessmann, L. (2008). The role of cognitive skills in economic development. Journal of Economic Literature, 46(3), 607-668. doi:10.1257/jel.46.3.607 International Association for the Evaluation of Educational Achievement. (2011). Trends in Mathematics and Science Study 2011. Retrieved from http://www.iea.nl/timss_2011.html Kaul, V., Bhattacharjea, S., Chaudhary, A. B., Ramanujan, P., Banerji, M., & Nanda, M. (2017). The India Early Childhood Education Impact Study. Retrieved from UNICEF: http://img.asercentre.org/docs/Research %20and%20Assessments/Current/Educat ion/Research%20Projects/IECEIStudyRe port2017.pdf Khan F. (1999). The social context of learning mathematics: Stepping beyond the cognitive framework. Mind, Culture and Activity, 6(4), 304-313 Kochetkova, E., & Brombacher, A. (2014). Ghana 2013 Early Grad Reading and Early Grade Mathematics Assessment: Report of findings. Retrieved from Research Triangle Park, NC: http://pdf.usaid.gov/pdf_docs/PA00MH MS.pdf MELQO Core Team, Technical Advisory Groups, & Steering Committee. (2017). Overview: MELQO Measuring Early Learning Quality and Outcomes. Retrieved from Paris, France: http://unesdoc.unesco.org/images/0024/ 002480/248053E.pdf Moll, L. C., Amanti, C., Neff, D., & González, N. (1992). Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms. Theory into Practice, 31(2), 132-141 Nasir, N. (2000). “Points ain’t everything”: Emergent goals and average and percent understandings in the play of basketball of African American students. Anthropology and Education Quarterly 31(3). 238-305 Nunes-Carraher, T., Carraher, D., & Schliemann, https://www.tcrecord.org/content.asp?contentid=16980) https://www.tcrecord.org/content.asp?contentid=16980) http://www/ http://img.asercentre.org/docs/Research%20and%20Assessments/Current/Education/Research%20Projects/IECEIStudyReport2017.pdf http://img.asercentre.org/docs/Research%20and%20Assessments/Current/Education/Research%20Projects/IECEIStudyReport2017.pdf http://img.asercentre.org/docs/Research%20and%20Assessments/Current/Education/Research%20Projects/IECEIStudyReport2017.pdf http://img.asercentre.org/docs/Research%20and%20Assessments/Current/Education/Research%20Projects/IECEIStudyReport2017.pdf http://pdf.usaid.gov/pdf_docs/PA00MHMS.pdf http://pdf.usaid.gov/pdf_docs/PA00MHMS.pdf http://unesdoc.unesco.org/images/0024/002480/248053E.pdf http://unesdoc.unesco.org/images/0024/002480/248053E.pdf Capturing children’s mathematical knowledge 123 A.D. (1987). Written and oral mathematics. Journal for Research in Mathematics Education, 18(2), 83-97 Nunes, T., Schliemann, A.D., & Carraher, D.W. (1993). Street mathematics and school mathematics. Cambridge University Press. PASEC. (2015). Education system performance in Francophone Sub-Saharan Africa: Competencies and learning factors in primary education. Dakar, Senegal: Programme d'Analyse des Systemes Educatifsde la CONFEMEN. Piper, B., Ralaingita, W., Akach, L., & King, S. (2016). Improving procedural and conceptual mathematics outcomes: Evidence from a randomised controlled trial in Kenya. Journal of Developmental Effectiveness, 8(3), 404-422. doi:10.1080/19439342.2016.1149502 Platas, L. M., Ketterlin-Geller, L. R., & Sitabkhan, Y. (2016). Using an assessment of early mathematical knowledge and skills to inform policy and practice: Examples from the early grade mathematics assessment. International Journal of Education in Mathematics, Science and Technology, 4(3), 163-173. Radford, L. (1997). On psychology, historical epistemology, and the teaching of mathematics: Towards a soci-cultural history of mathematics. For the Learning of Mathematics, 17(1), 26-33. Raikes, A., Britto, P. R., Yoshikawa, H., & Iruka, I. (2017). Children, youth and developmental science in the 2015-2030 Global Sustainable Development Goals. Social Policy Report, 30(3), 1-23. RTI International. (2014). EGMA Toolkit. Washington, DC: Research Triangle International. Saxe, G. (1991). Culture and cognitive development: Studies in mathematical understanding. NJ: Lawrence Erlbaum Associates, Inc. Sitabkhan, Y. (2009). The use of convenient values among young train vendors in Mumbai, India. Proceedings of the Episteme 3 Conference, (pp. 114-118). Mumbai, India: Homi Bhabha Center for Science Education, TIFR. Sitabkhan, Y. (2015). "¡Chalinas a 20 Pesos!": Children's Economic Ideas Developed through Selling. Mind, Culture, and Activity, 22(3), 269- 279. Taylor, E. V. (2009). The purchasing practice of low-income students: The relationship to mathematical development. The Journal of the Learning Sciences, 18(3), 370-415. Taylor, E.V. (2011). Supporting children’s mathematical understanding: Professional development focused on out-of-school practices. Journal of Mathematics Teacher Education, 15(4), 271-291. UNESCO. (2015). EFA Global Monitoring Report: Education for All 2000-2015: Achievements and Challenges. Retrieved from http://unesdoc.unesco.org/images/0023/ 002322/232205e.pdf Uwezo. (2015a). Are our children learning? Five stories on the state of education in Kenya in 2015 and beyond. Retrieved from http://www.uwezo.net/wp- content/uploads/2016/05/05-16-Kenya- small-size.pdf Uwezo. (2015b). Are our children learning? Five stories on the state of education in Uganda in 2015 and beyond. Retrieved from http://www.uwezo.net/wp- http://www.uwezo.net/wp-content/uploads/2016/05/05-16-Kenya-small-size.pdf http://www.uwezo.net/wp-content/uploads/2016/05/05-16-Kenya-small-size.pdf http://www.uwezo.net/wp-content/uploads/2016/05/05-16-Kenya-small-size.pdf http://www.uwezo.net/wp-content/uploads/2016/06/Uwezo2014-ALA-UG-FINAL-web.pdf 124 Global Education Review 5(3) content/uploads/2016/06/Uwezo2014- ALA-UG-FINAL-web.pdf Vygotsky, L.S. (1986). Thought and Language. Cambridge, MA: MIT Press. (Chap. 4, 6). Watts, T. W., Duncan, G. J., Siegler, R. S., & Davis-Kean, P. E. (2014). What's past is prologue: Relations between early mathematics knowledge and high school achievement. Educational Researcher, 43, 352-360. doi:10.3102/0013189X14553660 About the Author(s) Yasmin Sitabkhan, PhD, is a Senior Early Childhood Education Researcher and Advisor in RTI’s International Education Division. In her current role at RTI, she provides technical support to projects in low- and middle-income countries in early mathematics. Her research interests focus on children’s development of early mathematical concepts and instructional strategies to support learning in low- and middle-income contexts. Dr. Sitabkhan has a Ph.D. in Education from the University of California, Berkeley. Linda M. Platas, PhD, is the associate chair of the Child and Adolescent Development Department at SF State University. She has participated in the development of child assessment instruments including the Early Grades Math Assessment (EGMA) and the Measuring Early Learning Quality and Outcomes (MELQO) and served as an expert in mathematics and literacy development on many technical and policy groups. She is a member of the Development and Research in Early Math Education (DREME) Network. Leanne Ketterlin Geller, PhD, is the Texas Instruments Endowed Chair in Education and professor in the Simmons School of Education and Human Development at Southern Methodist University. Her research focuses on supporting student achievement in mathematics through developing technically rigorous formative assessment procedures and effective classroom practices. Her work emphasizes valid decision-making systems for students with diverse needs. http://www.uwezo.net/wp-content/uploads/2016/06/Uwezo2014-ALA-UG-FINAL-web.pdf http://www.uwezo.net/wp-content/uploads/2016/06/Uwezo2014-ALA-UG-FINAL-web.pdf