What goes on inside Latin American math and science classrooms 110 Global Education Review is a publication of The School of Education at Mercy College, New York. This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial 3.0 Unported License, permitting all non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. Citation: Naslund-Hadley, Emma, Loera Varela, Armando & Hepworth, Katelyn (2014). What goes on in Latin American Math and Science Classrooms: A video study of teaching practices. Global Education Review, 1 (3). 110-128. What Goes On Inside Latin American Math and Science Classrooms: A Video Study of Teaching Practices Emma Näslund-Hadley Inter-American Development Bank Armando Loera Varela Heurística Educative Katelyn Ann Hepworth Inter-American Development Bank Abstract Beyond common associated factors, such as teacher characteristics and socio-economic background of students, little is known about how student achievement in math and science is related to differences in the teaching approaches used in Latin American classrooms. This paper highlights the main findings of a qualitative study on cross-country differences in teaching practices in three Latin American countries. Of the three countries selected for the study, Paraguay and the Dominican Republic perform at the bottom of the regional comparative test, Second Regional Comparative and Explanatory Study (SERCE), and the Mexican state of Nuevo Leon is one of the top performers. Our findings, based on a large sample of videotape recordings from sixth-grade classrooms in the three countries, indicate that inquiry based instruction appears to be associated with higher levels of learning. Teachers who actively engage students in activities that promote analytical and critical-thinking skills and move beyond a procedural understanding may lead to better performance on the SERCE assessments. However, drill, practice, and memorization predominate in all three countries. Keywords: math and science education, videotape recordings, inquiry based instruction, teaching approaches, SERCE, Paraguay, Dominican Republic, Mexico Introduction In recent years scholars have devoted much attention to explaining differences in student achievement among countries. Some of the differences in achievement on international standardized tests, such as the Program for International Student Assessment (PISA), Trends in International Math and Science Study (TIMSS), or Progress in International Reading Literacy Study (PIRLS), can be attributed to the characteristics of the teachers, students and schools, e.g., years of experience and training of the teachers, socio-economic background of students, and condition of the school _________________________________ Corresponding Author: 1,2 Emma Naslund-Hadley, Inter-American Development Bank, 1300 New York Avenue, NW, Washington, DC 20577. Email: emman@iadb.org What goes on inside Latin American math and science classrooms 111 infrastructure (Baker, Goesling & LeTendre, 2002; Chiu, 2010). Chiu (2010) also found math achievement to be linked to less tangible factors, such as school discipline and student–teacher relationships. Others have sought to go inside classrooms to document instructional practices. The TIMSS video study was one of the first of its kind to analyze pedagogical approaches through large-scale classroom observations (Stigler, Gonzales, Kawanaka, Knoll & Serrano, 1999; Stigler, Gallimore & Hiebert, 2000; Hiebert, 2003; Roth et al., 2006). Recording math and science classes on videotape allowed researchers to thoroughly document content as well as teacher and student activities. The study provided possible explanations for cross-country differences on international examinations allowing policymakers and educators to gain a better sense of what was happening in classrooms, and how closely teachers adhered to the outlined curriculum. In Latin America and the Caribbean (LAC) not much is known about what is happening in math and science classrooms. What pedagogical approaches are used to teach math and science content? How effective are teaching practices in producing high levels of learning? These questions motivated our research study, which attempted to shed light on reasons for some of the differences in achievement among Latin American countries. Teaching approaches continue to change as we better understand how students learn and which teaching methods are associated with higher levels of learning. Studies have provided evidence that inquiry-based instruction practices that use hands-on activities to engage students in learning content are associated with increased learning, higher achievement, and greater student motivation in comparison with traditional instruction methods (Anderson, 2002; Furtak, Seidel, Iverson & Briggs, 2012). These findings suggest that at least some degree of inquiry-based instruction should be used to maximize math and science learning (Healy, 1990; Lowery, 1998; Colburn, 2000). This finding is supported by the TIMSS 1995 and 1999 video studies. Evidence from these studies showed higher levels of inquiry-based instruction in classrooms in countries that performed better on international tests. In the TIMSS 1995 study in math, there were major instructional differences in classrooms in Japan, the highest performing country in the study, compared to the other countries included. Using a video study, we analyzed to what extent inquiry-based instruction was being applied in classrooms in the Dominican Republic, Paraguay, and the Mexican state of Nuevo Leon, and if there were any relationships between the levels of inquiry-based instruction and student performance on the Second Regional Comparative and Explanatory Study (SERCE). In addition to coding the videos, we interviewed each teacher and allowed them to reflect on their own lesson based on the video recording. The interview covered issues related to the preparation and implementation of the lesson, the work environment, the teachers’ perception of the students’ performance during the lesson, and the overall quality of the lesson. The videos were also complemented with a questionnaire filled out by the teachers, which encompassed queries about the availability of didactic materials and science labs, as well perceived difficulty of teaching sixth grade science and mathematics. Background Math and Science Learning in Latin America Latin American students perform far below their peers in most developed countries in math and science. The poor performance of students in Latin America on international assessments has been well documented (Organisation for Economic Co-operation and Development [OECD], 2009; International Association for the Evaluation of Educational Achievement [IEA], 2007). Although performance on international assessments has improved in recent years, the 112 Global Education Review 1(3) 2009 results of the PISA and the 2011 results of the TIMSS demonstrate that Latin American students are still among the worst performers in both content area and skill development. Disparities in performance on assessments exist not only when comparing students in Latin America with students in other regions, but variation in performance persists from country to country within the region. Students in some countries perform far better than their peers in other countries within Latin America and the Caribbean (United Nations Educational, Scientific, and Cultural Organization- Latin American Laboratory for Assessment of the Quality of Education [UNESCO-LLECE], 2008). Table 1 demonstrates the variation that exists among countries in the results of the 2006 SERCE, which assessed the science and math skills of sixth-grade students in 14 countries and territories in Latin America and two participating countries from the Caribbean. The results are summarized in terms of the proportion of students that achieved each level of competency in math and science. In the Dominican Republic, more than 47 percent of sixth grade students that participated in the SERCE failed to achieve a level II competency. They were unable to solve problems that required multiplication or division, do addition with fractions, or recognize common geometric shapes. Very few children, less than 7 percent of students in the Dominican Republic, reached level III or IV. Paraguayan students performed better than their counterparts in the Dominican Republic, but the number of students achieving level II or above is well below the LAC average. In contrast, students in the Mexican state of Nuevo Leon ranked as some of the top performers on the SERCE. Less than 7 percent of students in Nuevo Leon achieved level I or below and a majority of the students scored above level II. In sixth-grade science, the situation is even more worrisome. The poor performance in science is a regional issue, as evidenced by the low proportion of students performing at level III or IV within the LAC average. More than half of the students in the Dominican Republic and Paraguay did not reach level II, indicating that they lack the skills to organize and compare information and classify living creatures according to predefined criteria. A greater proportion of students in Nuevo Leon achieved level II or above than in Paraguay or the Dominican Republic, but a large portion of sixth-graders, close to 34 percent, scored level I or below. Table 1 SERCE 2006: Proportion of students achieving competency levels (I-IV) in math and science (%) Math Science Performance Level LAC Dominican Republic Paraguay Nuevo Leon LAC Dominican Republic Paraguay Nuevo Leon Below I 1.48 5.69 3.85 0.34 5.18 14.29 7.20 2.59 I 13.91 41.79 21.00 6.29 38.72 62.82 46.18 30.98 II 40.82 45.43 46.50 29.35 42.24 21.50 38.11 47.78 III 32.35 6.85 23.91 40.66 11.40 1.37 7.52 16.38 IV 11.44 0.24 4.74 23.36 2.46 0.03 0.99 2.28 Source: UNESCO-LLECE 2008. What goes on inside Latin American math and science classrooms 113 Note: LAC = Latin America and the Caribbean. The results of the SERCE illustrate major learning gaps in math and science throughout the region. Some differences in achievement can be attributed to the quality of the teachers, the socio-economic background of the student or the characteristics of the school, but differences in these factors do not account for all variations in learning (Levin & Lockheed, 1993; UNESCO- LLECE, 2008). What happens in the classroom is a major contributor to learning; however, not much is known about the relationship between student achievement and teacher performance in Latin American classrooms. Low student performance on regional and international assessments leads us to question what is actually going on in the classroom and what teaching practices are being followed. A review of regional curriculum suggests that inquiry-based instruction should be the principal teaching approach used in the classroom (Valverde, 2009). In order to analyze the use of inquiry within Latin American and Caribbean classrooms, it was necessary to first identify common characteristics of an inquiry-based instructional approach. Inquiry-based Teaching Approaches Inquiry-based instruction is one of the most effective pedagogical approaches. Anderson (2002) asserts that using an inquiry-based or discovery approach supports higher achievement and greater student motivation. Inquiry-based approaches can also contribute to a significant increase in student conceptual learning, according to Furtak et al. (2012). There is no commonly agreed upon definition of inquiry-based instruction, but several scholars have suggested the use of some type of spectrum to classify different types of inquiry based teaching practices. Colburn (2000) distinguishes four types of inquiry-based instruction: structured inquiry; guided inquiry; open inquiry; and learning cycle. Each type provides a varying degree of inquiry based on the guidance provided by the teacher. Similar to the classification provided by Colburn, Furtak et al. (2012) also defines the type of inquiry instruction by the level of guidance provided by the teacher. Teacher-led instruction is most often characterized by lecture, practice, and drill. In comparison with inquiry-based instruction, several studies have found that traditional instruction approaches lead to lower levels of learning and motivation to learn science content (Chang & Mao, 1999). On the opposite end of the spectrum, though a student-led discovery approach allows students the freedom to guide the lesson based on interest and curiosity the approach may not increase levels of learning in comparison with teacher-led instruction. Studies have shown that without guidance from the teacher, activities do not automatically increase students’ levels of learning because it is harder for students to draw concrete conclusions based on student-led activities. A balance between teacher-led instruction and student-led discovery may produce the greatest learning outcomes. The guidance provided by the teacher can be further defined as an implicit or explicit approach with the inquiry-based method. Identifying instruction as implicit or explicit describes the way in which the teacher provides content to the students. With an implicit approach, students are not lectured or informed of the concepts before performing activities. Implicit instruction is based on the belief that students will learn through engaging in hands- on activities, but participation does not ensure students will learn the scientific concepts and theories implemented during the lesson (Khishfe & Abd-El-Khalick, 2002). Without explicitly discussing the material covered, the objective of 114 Global Education Review 1(3) the lesson can remain ambiguous. Often, ambiguity is misconstrued as inquiry and the lack of defined concepts does not contribute to students’ learning (Andrews, 2013). Khishfe and Abd-El-Khalick (2002) recommend an explicit- reflective approach that engages students in inquiry-based activities with planned reflective periods and discussion to make connections and highlight the important skills and concepts developed in the activities. Dialogue between the teacher and the students is encouraged to help students generate, develop, and justify explanations as part of the science activities (Furtak et al, 2012). Inquiry-based teaching practices are widely accepted as critical for students to develop scientific thinking skills; however, additional research is required to determine what degree of inquiry is most effective (Marzano, Pickering, & Pollack, 2001). Similar to questions about the effectiveness of inquiry-based instruction in science, there is much debate regarding procedural versus conceptual understanding in math (Skemp, 1987). Procedural understanding is defined as knowing how to obtain a correct answer without understanding the method used. Students memorize formulas and are able to produce an answer by inserting numbers into the equation; little interpretation of the process or reflection upon the meaning of the answer is necessary. The focus is on producing the correct answer, independent of understanding how and why students arrived at the answer. In contrast, conceptual understanding emphasizes the need to both solve the problem and understand how the process works. Some experts argue that a foundational framework of memorization of formulas and definitions is essential in progressing to more advanced complex concepts, but others have found that memorization of basic concepts impedes later meaningful learning because students initiated early into relying on memorization are less inclined to develop critical thinking skills (Pesek & Kirshner, 2000; Zacharos, 2006). Some memorization is necessary to master certain basic skills, such as multiplication tables and common subtraction. However, teachers should promote critical thinking approaches that result in not only learning concepts and formulas, but in understanding how formulas function, and what a correct answer means. Based on the literature reviewed, we focused our investigation on reviewing classroom activities and determining the use of inquiry-based instruction in math and science classes and the prevalence of procedural understanding in math. The most effective way to produce the evidence needed was to go into the classroom and film class lessons for further analysis. The Video-study In 2010 we filmed math and science classes in 291 schools in three countries. This study is the first large-scale systematic cross- country assessment of pedagogical processes in math and science classrooms in Latin America. In order to accurately describe math and science teaching in Paraguay, the Dominican Republic and the Mexican state of Nuevo Leon, we needed samples that could be said to be illustrative of instruction in each country, and comparable to performance on an internationally accepted assessment. Using the 2006 SERCE samples from the Dominican Republic, Paraguay, and Nuevo Leon, we drew a random subsample of 100 elementary schools in each country, covering more than 70 percent of the original sample of schools. In each school, we randomly selected one science and one math class offered at the sixth-grade level. Similar to the TIMSS video studies, the national samples are not statistically representative, but large enough to identify teaching patterns at the national level(Stigler et al., 2000). What goes on inside Latin American math and science classrooms 115 Most schools in the sample were urban (63 percent), with 37 percent of the sample from rural areas. In classifying the sample based on type of school administration, 80 percent were public schools, 13 percent were private and secular, and 7 percent were private schools affiliated with a religious institution. Overall, half of the teachers had university degrees, but the proportion of teachers with university degrees differed greatly by country. Only one- fifth of teachers in Paraguay held university degrees. A greater percentage of teachers in the Dominican Republic and Nuevo Leon, 82 percent and 40 percent respectively, had university degrees or higher levels of education. Not surprisingly, teachers who had a university degree worked in schools with higher SERCE achievement levels in both math and science. As expected, urban schools showed higher performance than rural schools; and private schools showed higher tests scores than public schools. Modeled after the well-known 1995 and 1999 TIMSS videotape studies of eighth-grade classrooms in Australia, the Czech Republic, Japan, Hong Kong, the Netherlands, Germany, and the United States, our video study filmed each teacher once (Stigler, Gonzales, Kawanaka, Knoll, & Serrano, 1999). Class lessons were recorded with two cameras, one focused on the teacher and the other on students. The lessons were analyzed using the TIMSS video studies of 1995 and 1999 coding frameworks. The TIMSS video 1995 study developed coding based on what the literature suggested were important components of quality instruction. The basic criterion considered in the initial coding was the opportunity to learn the content of the lesson. This concept included use of time, organization of the class; occurrence of outside interruption; the organization of interaction; activity segments; and management of content. The length of time devoted to each criterion was recorded and most criteria contained sub- categories (Stigler et al., 1999). The organization of the class was divided into four sub-categories: pre-lesson, activities, the lesson, and post- lesson. The interaction criteria classified lesson time as classwork or seatwork. Any time identified as “seatwork” was further categorized as individual, group work, or mixed. The activity segments were defined as setting-up, working on, sharing, or teacher talk-demonstration. Math content was classified as tasks, situations, principles, properties, or definitions, teacher alternative solutions, and student generated methods. Science content was coded based on the nature of the scientific topic, the type of science knowledge, the level of difficulty, and the modes of content development (Roth et al., 2006). The class lesson was further coded to define the type of discourse, either public or private talk, during the lesson. Coders used software called Videograph to quantify the occurrence of different classroom activities as a percentage of the lesson´s time. A team of local pedagogical experts and subject matter experts coded classroom practices in 594 mathematics and science lessons covering a total of 504 classroom hours. They reviewed a total of 2,489 math problems, considering 210 variables related to math lessons and 192 related to science lessons. The problems were coded based on a framework that determined the procedural complexity, the type of method used to solve the problem, if a solution method was repeated. Problems were identified as exercise or application, and as proof, verification, or derivation (Vincent & Stacey, 2008). Using the Videograph software and analysis by the pedagogical experts, we created a database that included 1,169 indicators. The video codification was complemented by 371 teacher and 296 principal questionnaires, providing information about school and classroom resources and personal characteristics, such as education and 116 Global Education Review 1(3) training in teaching science and mathematics. The questionnaire data was used to identify national patterns and internal differences on teacher´s general profiles and availability of teaching resources. Once the class was finished, the teachers were invited to watch their performance and answer questions in a video-recorded interview. These questions were related to lesson planning strategies; a self-assessment of their activities and the performance of the students; a discussion of alternative activities that could be implemented; the influence of existing resources in their decision making process; and if the presence of cameras and researchers in the classroom affected or disrupted the normal flow of the class lesson. Some students were also asked if the teacher´s performance on the class was “typical” of her or him. All of the teachers considered the recorded class as representative of their teaching style, almost all interviewed students on this regard agreed. In almost two thirds of the recorded lessons in Paraguay, the teachers used Guarani in some segments of their classes, the native official language; Spanish being the other official language. These segments were translated to Spanish. However, no ethnographic analysis has been made on these lessons or segments yet. The results of the study are strictly explorative and should not be used to draw any reliable conclusions about individual teachers, given the limited observation of only one lesson recorded for each educator. Filming may have encouraged teachers to perform at their very best or possibly it caused anxiety that may have affected instruction. However, the TIMSS video studies demonstrated that systematic observation of what goes on in the classroom can help identify shared practices, routines, and discourse that are common to an education system (Stigler et al, 1999). Some practices positively contribute to a student’s foundation of knowledge, while other practices may not support learning and might even hamper it. The TIMSS study observed eighth-grade classrooms; therefore, we cannot directly compare results from our study with that of the TIMSS video studies. However, in both studies, results were based on the same non-grade specific indicators. Therefore, the results allow for cross-country comparison of culturally-specific trends in teaching approaches. That is to say very specific differences in teaching practices were seen between Japan, the highest performer on the TIMSS examination, and the United States and Germany within the 1995 TIMMS video study. Based on the conclusions from the study, we often abstractly compare our results to what was observed in high performing countries, especially to Japan in math. No claim is made on the cultural relevance of the findings because the focus of the study was the contrasting patterns at the national level of the three participant sites in this study. Results The results of the study identified important possible relationships between learning, and content and methodology. Based on comparisons between the three countries and links made between results from the TIMSS video studies, we conclude that the method through which content is presented has a strong relationship with learning and the development of certain critical thinking skills, and the type of content and complexity of the content will also affect how well students capture and absorb math and science knowledge. All results are summarized in Table 2. What goes on inside Latin American math and science classrooms 117 Table 2 Results of the video study Dominican Republic Paraguay Nuevo Leon, Mexico Proportion of math instruction time spent on different types of mathematical thinking (%) Applying concepts 57 59 67 Practicing routine procedures 43 39 30 Inventing new solutions 0 2 3 Math lessons in which students and teachers present alternative solutions to math problems (%) No alternative solutions considered 94 94 82 Students present alternative solutions 3 3 14 Teachers present alternative solutions 3 3 4 Proportion of science lessons that focused on development of connections versus acquiring facts, definitions, and algorithms (%) Acquiring facts, definitions, and algorithms 96 97 69 Making connections 4 3 31 Proportion of science instruction time devoted to work in seats and practical activities (%) Seat work- whole class 20 22 28 Seat work- independent 32 46 28 Practical activities- whole class 44 26 40 Practical activities- independent 4 6 4 Proportion of math lesson time devoted to new and previous content (%) Reviewing previous content 82 68 28 Introducing new content 11 12 15 Practicing new content 7 20 57 Complexity of the math problems covered (%) Low 89 76 94 Moderate 9 21 5 High 2 3 1 Proportion of science lessons with different levels content complexity as compared to the national curriculum (%) Basic 82 91 83 Basic and challenging 17 8 17 Challenging 1 1 0 Sources of content used during science lessons (%) Teachers 71 64 54 Textbooks 23 6 26 Worksheets 4 26 7 Other sources 2 4 13 Source: Authors’ Construction Science 118 Global Education Review 1(3) The TIMSS video studies used a wide variety of indicators to assess the level of inquiry used in science instruction. One indicator categorizes lessons according to how scientific content is developed: by encouraging students to make connections among ideas, experiences, patterns, and explanations; or by acquiring facts, definitions, and algorithms. In the classrooms we filmed, lessons focused on memorization of scientific concepts and learning the history of science rather than doing science. Students engaged in practical activities in all three countries: the Dominican Republic (48 percent) and Nuevo Leon (44 percent), and Paraguay (32 percent). However, student opportunities for hands-on learning were severely limited as the teacher performed demonstrations of almost all of the practical work in front of the entire class and students were seldom given time to work through procedures and develop an understanding of concepts. The few lessons that contained practical independent work were restricted to confirming findings already given to them by their teacher. Students were asked to replicate an activity modeled by the teacher, or the teacher would ask the students to follow a procedure to arrive at an outcome that the teacher had previously introduced. Rather than asking students (for example) to formulate predictions about the density and mass of different materials and to design experiments to test their predictions, the teacher might tell students that copper is denser than aluminum, and then have the students confirm that this is the case. Two-thirds of the practical experiments in Paraguay were classified as confirming content. Only in 6 percent of the Paraguayan lessons did students explore a research question independently. The situation was not much better in the Dominican Republic (7 percent) and only slightly better in the state of Nuevo Leon (11 percent). Evidence suggests that students who are taught through memorization of formulas and taught the history of science instead of actually doing science have lower levels of learning. The high proportion of reliance on presenting facts and definitions and the low performance of the countries on the SERCE in science supports this evidence. Connecting classroom lessons to real life situations that students may encounter has been shown to increase interest in science and improve learning. Students understand the application of concepts when lessons are linked to their environment (Bouillion & Gomez, 2001). In 38 percent of Paraguayan science lessons connections were made to the everyday lives of students by discussing the relationship between scientific concepts and everyday experiences, using everyday examples, or addressing reasons for studying science in lectures or in whole-class discussion. Links between the science content and the lives of the student were made during fewer lessons in the Dominican Republic (30 percent) and the Mexican state of Nuevo Leon (26 percent). In the in-depth interviews with the teachers, they often discussed the importance of linking the lesson content to everyday life. However, they had trouble converting this theory into practice as the actual time devoted to real-life issues was very limited: 2 percent in the Dominican Republic and Paraguay, and 3 percent in Nuevo Leon. Often teachers asked the students to make connections as a pre-lesson activity, but failed to refer back to the connection students suggested or present the content through the lens of the of student connections. It is questionable whether making connections had a significant effect considering the method was used quite infrequently and for very short periods of time. In the countries of the TIMSS video study such links to the everyday lives of students were made in 74 percent of the lessons on average, using 13 percent of public speaking time. In the TIMSS video studies, if lessons provided at least one opportunity for students to study science-related content regardless of the source, the lesson was identified as an What goes on inside Latin American math and science classrooms 119 opportunity to learn science (Stigler et al, 1999). In countries, included in the TIMSS video study, 90 percent of lessons provided an opportunity to learn science content. The average number of lessons providing science content in the Latin American countries was 44 percent. More than half of lessons lacked any type of science content during the time allocated for science instruction. Instead, the focus was on procedures, without any explicit connection made to the intended content. Procedures are an important component of science learning; however, ideally all material should be connected to a broader theme in order for students to conceptualize the purpose of the procedure within the science content. Lessons were further assessed according to the level of complexity of the content. Based on categories used within the TIMSS 1999 video study, content was rated as basic or challenging. Components of the lesson were identified as challenging if they were above the sixth-grade level as determined by national curriculum standards and goals. Concepts rated as basic were defined to be those that were below the sixth-grade curriculum level. The complexity of science content observed in Paraguay corresponded to sixth-grade curriculum standards. Only 9 percent of observed science lessons included some content that was above grade level. In the Dominican Republic and Nuevo Leon, the proportion of lessons that included some challenging content was somewhat higher (17 percent). This is not surprising as curricula are often misaligned with national learning standards or not fully implemented to meet national standards (Vegas & Petrow, 2008). However, the lack of challenging activities in science is worrisome because curricula in Latin America and the Caribbean often do not meet international standards (Valverde, 2009). Therefore, it is unclear whether the portion of science lessons designated as challenging within the study would also be considered challenging when compared to international standards. The sources of content used are important, as they help determine how lessons are organized. If textbooks are available, they can help to structure lessons and remove pressure from the teacher to provide content. The use of many sources of content allows the teacher to act as a learning facilitator, as observed in Japanese eighth-grade science lessons in the 1999 TIMSS video study. In Japan, the teacher was the source of content only 22 percent of the time. In contrast, in Paraguay and the Dominican Republic, the primary sources of science content were the teacher (64 percent and 71 percent) and worksheets (26 percent and 4 percent). Textbooks were used more often in classrooms in Nuevo Leon (26 percent) and the Dominican Republic (23 percent) as compared to Paraguay, where textbooks were used in just 6 percent of science lessons. The lack of textbook use, especially in Paraguay is not surprising. Unlike Nuevo Leon, where nine out of ten students have their own science textbooks, there is a lack of access to textbooks in Paraguay in both math and science (UNESCO-LLECE, 2008). Little is known about the quality of textbooks and their impact on learning in the region. Given that the teacher is the dominant source of content in Paraguay and the Dominican Republic, it is distressing that the study observed significant gaps in the content knowledge of teachers. In 59 of the 100 Paraguayan science classes observed, the teacher committed at least one error. Similar rates of error were observed in the Dominican Republic and Nuevo Leon. Conceptual errors, ranging from a statement that sunlight causes the earth to rotate, to the incorrect labeling of parts of plants and the human body, represented the most frequent type of error, particularly in the area of reproductive health. Specific errors included misrepresentations of how diseases spread and the functioning of the reproductive system. Some errors reflected gender stereotypes, as was the case when students in one class were taught that drug and alcohol use by males tends to produce female offspring. 120 Global Education Review 1(3) Other errors were categorized as either procedural (omitting steps or taking them in the wrong order when manipulating data or conducting experiments), or factual (attributing historic scientific discoveries to the wrong inventor). In more than 90 percent of cases, neither the students nor the teacher noticed the error, and in the few cases where the teacher seemed to catch the error, they were reluctant to correct it. Equally as alarming, when students committed errors, teachers neither corrected them nor guided them to discover the error. This was the case in 47 percent of cases in Paraguay, 50 percent of cases in the Dominican Republic, and 53 percent of cases in Nuevo Leon. Math In the math lessons, teachers focused primarily on the presentation and repetition of math procedures. The remaining time was spent copying from the blackboard, doing drills and practice, and memorizing math concepts. Little to no time was spent on considering new solutions. Only 2 percent of the effective lesson time was used for activities that required critical thinking in Paraguay, and no time was devoted to inventing new solutions in the Dominican Republic. In Nuevo Leon, 30 percent of the time was devoted to practicing routine procedures, 67 percent was allocated for applying concepts, but only 3 percent of instruction time was used for inventing new solutions. The observations indicate a virtually exclusive focus on the development of procedural understanding. This focus on procedural understanding is very different than what was observed in eighth-grade classrooms in high-achieving countries in the TIMSS video studies. In Japan, students only spent 15 percent of lesson time applying concepts and used a larger proportion of the time, 44 percent, to invent new solutions (Stigler et al., 1999). Not surprisingly, the proportion of math problems for which either the student or teacher presented an alternative solution was extremely low. Development of alternative problem- solving methods is widely believed to be central to the development of conceptual math understanding (National Council of Teachers of Mathematics [NCTM], 2000; Hiebert, Carpenter, Fennema, Fuson, Wearne, & Murray, 1997). Only in a limited number of the Latin American classes that our team analyzed did students or teachers present alternative solutions. The teacher or the students discussed alternative solution methods in just 3 percent of math problems in Paraguay. In classrooms in Nuevo Leon, the presentation of alternative solutions was more frequent; in 14 percent of lessons alternative solutions were presented by students and in 4 percent of lessons by the teacher, but those figures are still significantly low in comparison to other countries. In the TIMSS video studies, teachers in Japan frequently encouraged students to identify alternative solution methods to math problems (42 percent of the lessons and 17 percent of all math problems) (Stigler et al., 1999). The lack of opportunities to invent new solutions during instruction and present alternative solutions for practice problems emphasizes the reliance on procedural understanding in math classes in Latin America. Students are not encouraged to expand their critical thinking skills. As was done in science, several indicators were created to define students’ opportunities to learn math content. Based on trends seen within the TIMSS studies, we were interested in evaluating the emphasis on new content and previously studied content. The TIMSS 1995 video study found that eighth-grade classrooms in high-achieving countries, such as Hong Kong and Japan, dedicated close to 80 percent of the instruction time to new content (Stigler et al., 1999). Lesson time was identified as introducing new content, practicing new content, or reviewing previous content. In Paraguay and the Dominican Republic, a small portion of lesson time was allocated to the introduction or What goes on inside Latin American math and science classrooms 121 practice of new content, 32 and 18 percent, respectively. In the Dominican Republic, very little time was used for practicing new concepts, only 7 percent. In contrast, students in Nuevo Leon spent the majority of math lesson time focusing on new content. Introducing new content accounted for 15 percent of lesson time, while 57 percent of the time was allocated to practicing the new content through group work or individual work solving problems. Students in Nuevo Leon spent only 15 percent of time in class working on content that they had looked at in previous lessons. Another important indicator of content is the complexity of the practice problems assigned to students. The number of decisions students need to make and steps or sub-problems necessary to arrive at a solution, provides the criteria for the degree of complexity of the problem. Problems are characterized as low, moderate, or high complexity based on the classification system contained in the 1999 TIMSS video study. Low-complexity math problems involve few decisions and do not contain any sub-problems. Problems of moderate complexity are those that require the student to make more than four decisions with the possibility of either no sub-problem or one sub-problem (Hiebert, 2003). Problems that involve four or more decisions and two or more sub-problems are classified as high complexity. In all three countries, low complexity math problems predominated. Low complexity problems accounted for 76 percent of the problems covered in Paraguay. In the Dominican Republic and Nuevo Leon, the proportion of low-complexity math problems was even higher, 89 percent and 94 percent respectively. All countries lacked high- complexity problems. Only one percent of problems were defined as high-complexity and 5 percent were assessed as moderate-complexity in Nuevo Leon, the highest performer on the SERCE exam. In seven relatively high- performing countries included in the 1999 TIMSS video study, 40 percent of problems were categorized as high complexity (Hiebert, 2003). The complexity of the problems is important because if activities are too easy students will not have the opportunity to develop the higher-level critical thinking skills (Colburn, 2000). However, if the problems are too difficult, students will not effectively learn the content. Teachers need to balance the levels of complexity to ensure that students are given ample opportunity to learn the content presented and also to develop critical skills using the content. Discussion This study shifts the conversation from teacher characteristics to what is going on inside Latin American classrooms and how teachers approach math and science in their daily practice. The results of the study provide important insight into the strengths and weakness of the pedagogical approaches used in three Latin American countries. A combination of drill, practice, and memorization continue to be the primary method of teaching. Instructional approaches reflected traditional teaching methods which sought to provide a procedural understanding of content through the memorizations of facts and formulas. Few teachers in the sample made a concerted effort to actively engage students in hands-on science activities that provided opportunities to cultivate important analytical and critical thinking skills. This is an issue that policy makers in the region will want to address as the literature suggests that instructional practices have a strong relationship with performance on educational assessments (Stigler et al, 1999). In Nuevo Leon, teachers implemented inquiry-based instructional approaches on a more frequent basis and in a higher proportion of classrooms. These differences may account for some of the disparity in performance on assessments between Nuevo Leon, and the Dominican Republic and Paraguay. Students in 122 Global Education Review 1(3) Nuevo Leon, Mexico, performed better on the SERCE exam than students in either Paraguay or the Dominican Republic. While the study indicated many similarities in teaching practices used in classrooms in Paraguay, the Dominican Republic, and Nuevo Leon, it is important to highlight the specific differences. In the Dominican Republic and Paraguay we observed a stronger focus on drill and memorization with limited opportunities to engage in activities that stimulated the development of important critical thinking skills of students when compared to Nuevo Leon. In Nuevo Leon, a greater proportion of lesson time was devoted to introducing and practicing new content. The role of the teacher in the classroom also indicated a major difference between common practices within the countries. In Paraguay and the Dominican Republic, the teacher was the main source of content and knowledge. There was little room for students to acquire knowledge through inquiry or discovery. This was also true for students in Nuevo Leon, but to a lesser extent than in Paraguay and the Dominican Republic. A lack of interactive and inquiry-based practices in the classroom was the main observation provided by the study about classes in Paraguay and the Dominican Republic, and to a lesser degree in Nuevo Leon. The teacher in-depth interviews provide some insight as to why inquiry-based approaches are not being implemented. Observations provided by the teachers demonstrate that teachers vaguely understand the importance of an interactive approach to math and science instruction. Several teachers noted that students are more excited and motivated when the teachers use games or experiments in lessons; however, they also shared certain obstacles to incorporating an inquiry-based approach into their classroom instruction practices. In science classes, students spent the majority of the class time memorizing facts and learning the history of science rather than performing scientific experiments. Even when students were given the opportunity to participate in interactive activities, the emphasis was placed on the procedure. Either the teacher performed the experiment and asked children to observe the result, or students were asked to perform a procedure to confirm an outcome that the teacher had already described. Students were not expected to think critically about the problem and formulate a hypothesis. Several teachers in Paraguay cited a lack of science materials as the reason they did not perform more experiments in science classes. However, the initial results of an ongoing experimental pilot in Argentina –contrasting two models of inquiry-based instruction– reveal that students who use simple classroom-based science kits learn as much as students who have access to more-sophisticated science materials and equipment (Author, Cabrol, & Ibarran, 2009), thus demonstrating that hands-on science activities are not dependent upon expensive materials and advanced science labs. Basic materials and equipment may be just as effective in teaching content as more expensive tools. Likewise in math classes, students were expected to memorize formulas and procedures, with little innovation in how concepts were presented to students. One teacher noted that she preferred to walk the entire class through several math examples step-by-step in order to memorize the formula. By completing several examples as a class, students would recognize the concept as familiar when doing homework problems. The classroom practices observed stands in sharp contrast with the literature on good instructional methods for learning mathematical and scientific reasoning and problem solving (Colburn, 2000; Anderson, 2002; Furtak, et al., 2012; Andrews, 2013 ). Some teachers emphasized that presenting the content through a traditional approach was the most effective method for student learning and not by providing extra time for activities. What goes on inside Latin American math and science classrooms 123 Other teachers stated that copying information was of the utmost importance because students needed notes to study for exams and, therefore, they did not attempt the activity. Several teachers expressed that time limitations were a major constraint in planning for science experiments and more interactive lessons. A teacher in the Dominican Republic discussed his inability to be flexible with time explaining that he only had forty-five minutes with the students before the end of the period and there is no allowance for experiments to go beyond the allotted class time. Other teachers adapted lessons plans to meet time constraints, but noted that doing so can potentially hamper the learning process. They stated that teachers need to provide students with ample time to digest information, work through any confusion, and allow students to make connections with the content. One teacher noted that she often performed the calculations for students when her objective was to teach a formula because students took too much time in completing sub-problems, specifically multiplication. This practice can be detrimental to learning as evidenced by students’ difficulty in completing problems on their own. The teacher remarked that students were unable to complete practice problems because they were confused by the added steps within the sub-problems, or they performed the calculations incorrectly and could not arrive at the correct answer. The literature indicates that providing answers to the students does not help them understand the concepts or give them the support to solve problems independently. For example, a recent meta-analysis of inquiry-based teaching found that when students are able to answer questions or are guided to a solution by the teachers, it helps to build their self- confidence and interest in math and science (Furtak et al., 2012). Definitions of scientific literacy often include the ability to identify and weigh alternative explanations of events (American Association of the Advancement of Science, 1993). It is therefore important to allow students to consider alternative solutions. Developing alternative solutions encourages students to think critically about the content and other knowledge they possess to find a solution to the problem. In one classroom, a student suggested a different method for solving a math problem. Initially, the teacher reprimanded the student for not following the formula presented and for not working diligently with her group. After checking the work, the teacher noted that the answer was correct, but instructed the student to focus on the group’s work and present the method the group used that followed the intended formula. Opposing the presentation of the alternative solution restricted an opportunity to develop critical thinking skills and could have damaged the student’s interest in math. The in-depth interviews also illustrated that while teachers are aware that students enjoy practical activities, teachers may not understand what type of practical activities promote learning. In multiple classrooms, drawing was used as the main activity to reinforce the concept taught during the class. In a math class in the Dominican Republic, students were to draw portions of fruit that represented the fraction they were assigned. Within a science lesson, students were instructed to draw the endangered species as projected from a slide in a classroom in Nuevo Leon. The teacher defended the activity because it encouraged students to pay attention and look at the slides the teacher presented. Students seemed engaged in both activities; however, it is unknown whether the drawing activities helped the children to learn the intended content. The quality of instruction also extends to classroom discourse. Many teachers begin lessons with a class discussion about the content, possibly asking the children to connect the topic to their everyday lives. Although it is positive that teachers are encouraging this type of interaction, the quality of discourse is important for learning. Asking multiple questions does not mean the teacher is 124 Global Education Review 1(3) providing interactive feedback (Smith & Higgins, 2006), suggesting that questions should invite students to elaborate and discuss complete ideas. These issues can be targeted through comprehensive training and practice of an inquiry-based approach. We found teachers were the main providers of math and science content. However, as demonstrated within the study, there are clear gaps in the knowledge of teachers. Research about the possible long-term negative effects of content errors during classroom instruction is limited, but we can assume it could have a detrimental effect on students’ future learning. The use of textbooks as the primary source of content could ameliorate the issue of teacher error. Studies show that the use of textbooks can have a large impact on the impact of student learning(Vegas & Petrow, 2008). Especially in Latin America, the quality of the textbooks provided is still relatively unknown. Many education systems in the region have undergone important reforms in the past two decades, but the textbooks provided to students may not reflect the changes made (Vegas & Petrow, 2008). Therefore, further research is necessary to establish that textbooks are high-quality; otherwise, they may not be any better than the content teachers provide. The purpose of our study was to explore what is actually going on inside classrooms and to provide insight into how certain pedagogical approaches are associated with learning outcomes. We noticed specific differences in teaching practices in different countries that may have affected student performance on regional assessments. It is our hope that our findings will stimulate dialogue and inspire educators and policymakers to design reforms and programs that improve students’ opportunities to learn math and science. Notes 1. Support for this research was provided by the Inter-American Development Bank. 2. The opinions expressed in this paper are those of the authors and do not necessarily reflect the views of the Inter-American Development Bank, its Board of Directors, or the countries they represent. References American Association for the Advancement of Science (1993). Benchmarks for Science Literacy. New York: Oxford University Press. Anderson, R. D. (2002). Reforming Science Teaching: What Research says about Inquiry. Journal of Science Teacher Education, 13(1), 1-12. Andrews, P. (2013). Finnish Mathematics Teaching from a Reform Perspective: A Video-Based Case-Study Analysis. Comparative Education Review, 57(2), 189-211. Baker, D. P., Goesling, B., & LeTendre, G. K. (2002). Socioeconomic Status, School Quality, and National Economic Development: A Cross-National Analysis of the "Heyneman-Loxely Effect" on Mathematics and Science Achievement. Comparative Education Review, 46(3), 291-312. Bouillion, L. M., & Gomez, L. M. (2001). Connecting School and Community with Science Learning: Real World Problems and School-Community Partnerships as Contextual Scaffolds. Journal of Research in Science Teaching, 38(8), 878-898. Chang, C.-Y., & Mao, S.-L. (1999). Comparison of Taiwan science students' outcomes with inquiry-group versus traditional instruction. The Journal of Education Research, 92(6), 340-346. Chiu, M. M. (2010). Effects of inequality, family and school on mathematics achievement: country and student differences. Social Forces, 88(4), 1645-1676. Colburn, A. (2000). An inquiry primer. Science Scope, 23(6), 42-44. Furtak, E., Seidel, T., Iverson, H., & Briggs, D. C. (2012, September). Experimental and Quasi-Experimental Studies of Inquiry-Based Science Teaching: A Meta- Analysis. Review of Educational Research, 82(3), 300-329. Healy, J. M. (1990). Endangered Minds: Why Our Children Don’t Think and What We Can Do About It. New York: Simon & Schuster. Hiebert, J. (2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 video study. DIANE Publishing. Hiebert, J., Carpenter, T. P., Fennema, E., Fuson, K. C., Wearne, D., & Murray, H. (1997). Making Sense: What goes on inside Latin American math and science classrooms 125 Teaching and Learning Mathematics with Understanding. Portsmouth, NH: Heinemann. International Association for the Evaluation of Educational Achievement [IEA]. (2007). Trends in Mathematics and Science Studies (TIMSS). Retrieved from http://timss.bc.edu/TIMSS2007/idb_ug.html Khishfe, R., & Abd-El-Khalick, F. (2002). Influence of explicit and reflective versus implicit inquiry- oriented instruction on sixth graders' views of nature of science. Journal of research in science teaching, 39(7), 551-578. Levin, H. M., & Lockheed, M. (1993). Effective Schools in Developing Countries. London: Falmer. Lowery, L. (1998) The Biological Basis of Thinking and Learning. Berkeley: University of California. Marzano, R. J., Pickering, D. J., & Pollack, J. E. (2001). Classroom Instruction That Works: Research-Based Strategies for Increasing Student Achievement. Naslund-Hadley, E., Cabrol, M., & Ibarran, P. (2009). Beyong Chalk and Talk: Experimental Math and Science Education in Argentina. Washington, DC: IDB. National Council of Teachers of Mathematics [NCTM]. (2000). Principles and Standards for School Mathematics. Reston, VA. Organisation for Economic Co-operation and Development [OECD]. (2009). Program for International Student Assessment. Retrieved from www.oecd.org/edu/pisa/2009 Pesek, D., & Kirshner, D. (2000). Interference of Instrumental Instruction in Subsequent Relational Learning. Journal for Research in Mathematics Education, 31(5), 524-40. Roth, K. J., Druker, S. L., Garnier, H. E., Lemmens, M., Chen, C., Kawanaka, T., et al. (2006). Teaching Science in Five Countries: Results from the TIMSS 1999 Video Study. Statistical Analysis Report. NCES 2006-011. National Center for Education Statistics. Skemp, R. R. (1987). The Psychology of Learning Mathematics: Expanded American Edition. Hillsdale, NJ: Psychology Press. Smith, H., & Higgins, S. (2006). Opening classroom interaction: the importance of feedback. Cambridge Journal of Education, 36(4), 485-502. Stigler, J. W., Gallimore, R., & Hiebert, J. (2000). Using video surveys to compare classrooms and teaching across cultures; Examples and lessons from the TIMSS video studies. Educational Psychologist, 35(2), 87-100. Stigler, J. W., Gonzales, P., Kawanaka, T., Knoll, S., & Serrano, A. (1999). The TIMSS Videotape Classroom Study: Methods from an Exploratory Research Project on Eighth-Grade Mathematics Instruction in Germany, Japan, and the United States. Washington, DC: National Center for Education Statistics. United Nations Educational, Scientific, and Cultural Organization- Latin American Laboratory for Assessment of the Quality of Education [UNESCO- LLECE]. (2008). Los Aprendizajes de los Estudiantes de América Latina y el Caribe. Santiago, Chile: OREALC/UNESCO. Valverde, G. A. (2009). Estandardes y evaluacion. In S. Schwartzaman, & C. Cox (Eds.), Polticas, Educativas y Cohesion Social en America Latina. Santiago, Chile: Coleccion Cieplan. Vegas, E., & Petrow, J. (2008). Raising Student Learning in Latin America. Washington, DC: The World Bank. Vincent, J., & Stacey, K. (2008). Do Mathematics Textbooks Cultivate Shallow Teaching? Applying the TIMSS Video Study Criteria to Australian Eight-grade Mathematics Textbooks. Mathematics Education Research Journal, 20(1), 82-107. Zacharos, K. (2006). Prevailing Education Practices for Area Measurement and Students' Failures in Measuring Areas. Journal of Mathematical Behaviour, 25, 224- 39. About the Author(s) Emma Naslund-Hadley is Lead Education Specialist at the Inter-American Development Bank. She manages the Bank’s efforts to improve mathematics and natural science education in Latin America and the Caribbean. Armando Loera Varela is director of Huerista Educativa consulting group. His areas of interest include teacher performance and school improvement policies. Katelyn Hepworth is a Research and Evaluation Officer at the Bernard Van Leer Foundation. Her interests include initial and primary education in Latin America http://timss.bc.edu/TIMSS2007/idb_ug.html 126 Global Education Review 1(3) Appendices Open Ended Teacher Interview Guide Preparation of Lesson. What was the purpose of your lesson? Please describe how you prepared the lesson. Did you prepare it differently than you usually do? If you used a lesson plan, would you please share it? How much did the lesson you prepared differ from the lesson you delivered? Why do you think that there was a difference (or lack thereof)? Delivery of Lesson. What's your own reaction to your delivery of the lesson? What's your reaction to the performance of your students? Would you please explain the logic behind the lesson: Why did you initiate it the way you did? Why did you choose to organize the students the way you did? How did you select which students to address? Why did you choose to conclude the lesson in the way you did? Would you say that this was a typical lesson? Teaching Environment. How would you say that the classroom environment facilitated or hampered the delivery of your lesson: The type and state of furniture? The noise level? The lighting? Availability of materials and equipment? General School characteristics? The school community? Student Activities. What lesson activity do you consider most interesting for the students? What activity do you think was least interesting? What do you think that the students learnt from your lesson? Do you consider that you accomplished the objective of your lesson? General Self-Appraisal of the Lesson. What does a great math/science lesson look like? How would you classify your performance during the lesson? How much do you consider that the presence of cameras influenced your delivery of the lesson? Seeing the video, is there anything that you wish you would have done differently during the lesson? Questionnaire Sixth-grade Teachers School: School code: 1. Age 2. Highest education level attained. Mark only one: Primary education; secondary education; technical non university; pedagogical non university; university degree; graduate studies; Other 3. How many years of teaching experience do you have? 4. How many years have you taught sixth grade? 5. Since you started teaching, how many Science teacher training courses have you taken on how to teach Science? 6. Since you started teaching, how many Math teacher training courses have you taken? 7. On a scale from 1 to 10 (1 being extremely easy and 10 being extremely difficult) how would you rate the difficulty of teaching sixth grade Science? What goes on inside Latin American math and science classrooms 127 8. On a scale from 1 to 10 (1 being extremely easy and 10 being extremely difficult) how would you rate the difficulty of teaching sixth grade Math? 9. Did you partake in the 2006 SERCE study? 10. How many students are there in your sixth grade classroom? 11. Which of the following materials are available in your classroom and with what frequency are they used by your sixth grade students in mathematics? a. Mathematics textbooks. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson b. Mathematics Workbooks. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson c. Counting frame. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson d. Logic blocks. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson e. Cuisenaire rods. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson f. Multi-base Materials. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson g. Tangram. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson h. Calculator. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson i. Geo-board with rubber bands. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson j. Mathematical manipulatives. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson 12. Which of the following materials are available in your classroom and with what frequency are they used by your sixth grade students in science? k. Science textbooks. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson l. Mathematic workbooks. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson m. Books about science experiments. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson n. Encyclopedias. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson o. Atlas. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson p. Magazines. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson q. Terrestrial Globe. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson r. Prints and/or maps. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson s. Magnifiers and/or scales. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson t. Microscope. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson u. Science manipulatives. Availability: yes/no Frequency of Use: Never/some lessons/most lessons/every lesson 13. Do you have access to a computer to teach sixth-grade? Yes/no How many? 14. Do you have access to a science lab to teach sixth-grade? Yes/No Hours of weekly use? 128 Global Education Review 1(3)