This study is intended to understand teaching quality of English student teachers when they conduct their teaching practicum. Teaching quality is conceptualized based on the principles of effective teaching resulted by teacher effectiveness studies. Thes IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 6 20 Years of TIMSS: Lessons for Indonesia THOMAS F. LUSCHEI 1 Abstract The Trends in International Mathematics and Science Study (TIMSS) has been administered to countries around the world every four years since 1995, generating rich cross-national data on school resources and student achievement that can inform policy and classroom practice. Although Indonesia has participated in every round of TIMSS, there is very little published research in English drawing educational lessons from these data for Indonesia. In this article, I described Indonesia’s performance on TIMSS over time and relative to other countries. Reviewing TIMSS reports on mathematics and science, I offered both positive and negative lessons from Indonesia’s participation in TIMSS since 1995. Keywords TIMSS, Indonesia, education, math, science 1 School of Educational Studies, Claremont Graduate University, 150 E 10th St. Claremont, CA 91711, USA; thomas.luschei@cgu.edu IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 7 Introduction In 2016, the International Association for the Evaluation of Educational Achievement (IEA) issued a report drawing lessons from 20 years of data generated by the Trends in International Mathematics and Science Study (TIMSS), a cross-national assessment of students’ knowledge in mathematics and science in the fourth and eighth grades. At the beginning of 1995, the IEA has administered TIMSS every four years with the most recent round occurred in 2015. Drawing on 20 years of student test score data and information from surveys of school leaders and teachers across dozens of countries, the IEA’s 2016 report offers insights related to trends in cross-national educational performance, curriculum, instruction, the distribution of student achievement, and students’ attitudes about mathematics. The IEA’s 2016 20 Years of TIMSS reports offers good news and bad news for many countries participated consistently in TIMSS since 1995. Several East Asian countries- Hong Kong, Japan, Singapore, and South Korea- had consistently high performance across 20 years of TIMSS assessments. Three of these countries actually improved their performance in fourth and eighth grade math and science from 1995 to 2015 (Japan’s performance in eighth grade math remained unchanged during this period) (Mullis et al., 2016). On the other hand, “the performance of Dutch fourth graders declined in both math and science, while Norway declined in fourth grade science and eighth grade math and science. Similarly, math and science performance are declined among eighth graders in both Hungary and Sweden (Mullis et al, 2016, p. 11).” In terms of the distribution of student achievement, the IEA’s report identifies many countries increased overall performance at the same time they reduced differences in achievement between low and high performing students. Most prominently, Portugal improved its overall performance in mathematics by 99 scale score points with low achievers improved their performance by 123 points and high achievers improved by 78 points (Mullis et al., 2016). Like many of the countries discussed in the IEA’s 20 Years of TIMSS report, Indonesia has participated consistently in TIMSS since 1995. What lessons does the report offer for Indonesia? It is almost none. The IEA’s 2016 report mentions Indonesia only twice in the following: Exhibit 10, page 19; Indonesia is listed as one of eight countries where girls had higher math achievement than boys in 2015. Page 29; in a section on problem solving and inquiry in mathematics and science curricula, the report observed that Indonesia has a new curriculum, introduced in 2013, includes observing, questioning, exploring, associating, and communication. Surely, participation in 20 years of TIMSS assessments must offer more lessons related to student performance and equity in Indonesia, the world’s fourth largest country. Unfortunately, due to idiosyncrasies in Indonesia’s TIMSS participation, discerning 20-year trends presents somewhat of a challenge that the IEA’s report did not take on. To further exacerbate the lack of TIMSS-based lessons for Indonesia, there has been relatively little peer- reviewed research published in English that used the TIMSS data to draw lessons for or from Indonesia. 1 1 A Google Scholar search for the terms “Indonesia” and “TIMSS” in the title yielded only 15 results. Only 5 of these were in English. IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 8 In this article, I attempted to shed some light on the question of what we can learn from Indonesia’s participation in TIMSS. I review Indonesia’s performance on each TIMSS assessment since 1995, paying special attention to the key dimensions; participation in TIMSS, student performance, relative performance to other nations in Southeast Asia, gender gaps in performance, students’ attitudes on math, and school climate and safety. After briefly reviewing related literature, I described the results of my review of Indonesia’s performance on TIMSS, offered both good and bad news for Indonesia. I concluded with a call for greater examination of the TIMSS data to draw lessons for Indonesia. Literature Review Despite a lack of published research pertaining to Indonesia’s TIMSS performance, the country’s participation in TIMSS and other international assessments nonetheless represents an important signal of interest on the part of Indonesian policy makers and educators to understand Indonesia’s educational performance from an international perspective. In this brief review of literature, I described why participation in international assessments is important. Further, I described TIMSS, and I discussed some key international lessons from 20 years of TIMSS assessments, as reported by the IEA. The value of international assessments From the perspective of a nation’s educational policy makers and educators, international assessments like the IEA’s TIMSS and Progress in Early Literacy Study (PIRLS), as well as the OECD’s Program for International Student Assessment (PISA), it serves several purposes. To begin, these assessments provide an assessment of student learning that can be compared to international averages and regional neighbors which allow the development of benchmarks to gauge educational performance and progress. Relative performance on international assessments can either provide educational “bragging rights” for countries that do well, or serve as motivation to catch and surpass neighbors that perform better. Student performance data also allow the identification and comparison of achievement gaps between different student groups, such as between boys and girls, poor and wealthy children, and students with different ethnic, racial, or language backgrounds. In addition to collecting student performance data using common instruments and procedures, international assessment efforts also collect information on school systems, schools, teachers, and students themselves. As a result, data from these assessments can be used to compare educational resources and contexts, as well as to measure students’ self-confidence and attitudes about school. Furthermore, these data can be used to answer a number of key descriptive questions, such as whether teachers or school leaders are distributed equitably across students, schools, or regions (Chudgar & Luschei, 2016). When analyzed in a multivariate framework, these data can also provide insights into whether certain school inputs are positively related to student performance, and whether relationships between inputs and outcomes vary within and across nations. As an example, a long line of research in the field of international comparative education has used data from international assessments to examine whether school inputs have a greater impact in lower- income contexts, where students have fewer resources in the home. Although the evidence on this question is mixed, there appears to be a stronger relationship between school resources and student achievement in contexts where resources are scarcer and more skewed toward the IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 9 wealthy (Chudgar & Luschei, 2009; Heyneman & Loxley, 1983). Although this research has focused on cross-national comparisons, researchers could use international assessment data to examine within- country relationships to identify contexts where educational investments might yield greater impacts in order to increase the efficiency of these investments. Finally, international assessment data allow researchers to examine the impact of cross- national variation in policy contexts that vary little within a given country (Chudgar & Luschei, 2014). For example, it is difficult for researchers to examine the impact of educational decentralization within a country because in most case, the education system is either centralized or decentralized. However, examination across countries allows researchers to examine a variety of systems that fall along the spectrum of centralization to see how these countries compare along a range of educational outcomes. Besides, there are also disadvantages in participating and conducting research related to international assessments. First, participation in assessments like PISA and TIMSS requires substantial resources in terms of financial investment and technical capacity. As a result, very few truly low-income countries participate in these assessments which limit the global generalizability of the data. Further, because these assessments do not track students longitudinally, researchers cannot assess within-cohort progress of students. The data also do not lend themselves easily to research that can draw causal inferences about the relationship between educational inputs and student outcomes. Despite these limitations however, the benefits of participating in international assessments outweigh the costs, especially if the resulting data are used to inform decisions about educational policy and practice. 20 years of TIMSS The IEA pioneered the use of international assessments in the 1960s with the First International Study of Mathematics in 1964 and the Second International Mathematics Study in 1980-1982, along with the First International Science Study in 1970-71 and Second International Science Study in 1983-84. Although these studies primarily compared educational performance in Europe and North America, the IEA’s Third International Mathematics and Science Study (TIMSS) in 1995 included 41 educational systems or countries spanning 6 continents, participating across third, fourth, seventh, and eighth grades, as well as the final year of secondary school. 2 The 1999 TIMSS included 38 countries or education systems and tested students only in the eighth grade. Since then, TIMSS has been administered every four years in both fourth and eighth grades, witnessing consistent and growing participation. In 2015, 57 countries or education systems, along with seven benchmarking participants (i.e., states or provinces) participated in either fourth or eighth grade, or both. Of the dozens of countries participated in TIMSS since 1995, the IEA’s 2016 20 Years of TIMSS report identifies 17countries that have participated and released data at the fourth grade level every four years between 1995 and 2015 (1995, 1999, 2003, 2007, 2011, and 2015). With a few exceptions, these countries have improved their educational performance. Of the 17 countries, 14 improved their mathematics performance in fourth grade between 1995 and 2015, while 2 countries decreased in achievement (Czech Republic and Netherlands). In the fourth grade science, 11 countries improved, while 2 countries 2 Over time, the name of the assessment was changed to the Trends in International Mathematics Science and Study. IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 10 declined (Netherlands and Norway). At the 8 th grade level, 16 countries participated in each round of TIMSS from 1995 to 2015. In both mathematics and science, 9 countries improved their performance, 3 countries had lower achievement, and average student achievement did not change in 4 countries. In 2015, more students than previous succeeded in reaching the most difficult benchmarks of achievement on math and science tests. The IEA’s 2016 report also describes key trends in achievement equity from 1995 to 2015. Gender gaps between girls and boys generally narrowed during this period. Additionally, while the number of countries where boys out performed girls decreased, the number of countries where girls outperformed boys increased. In the fourth grade math and science, gaps between low performers and high performers decreased, suggesting an improvement in achievement equity from 1995 to 2015 (Mullis, Martin, & Loveless, 2016). Indonesia’s participation in TIMSS: Lessons learned Table 1 illustrates Indonesia’s participation and performance on TIMSS from 1995 to 2015. Although Indonesian eighth graders participated in the 1995 assessment in grades 4 and 8, the data from Indonesia were not reported by IEA. According to the IEA’s report, “Indonesia and Italy were unable to complete the steps necessary for their data to appear in this report (Mullis et al., 1997, p. 11).” Indonesia participated in all rounds followed by, and data from these rounds were reported. In each round except 2015, Indonesia participated in Grade 8 only. In 2015, Indonesia participated in Grade 4 only. In the following, I discuss salient results in terms of participation in TIMSS, student performance levels, relative performance to other nations in Southeast Asia, gender gaps in performance, students’ attitudes about math, and school climate and safety. Table 1. Selected results from Indonesia’s participation in TIMSS, 1995-2015 Year & Subject Grade Score (Rank) Performance in SE Asia Gender Gaps Students’ Valuing of Subject Students’ Self-Confidence in Subject Students’ Perceptions of School Safety 19951 4 & 8 n/a n/a n/a n/a n/a n/a 19992 Mathema tics 8 403 (34/38) below Malaysia & Thailand (SS); above Philippines (SS) Boys scored 5 points higher (NS) Like a lot: 22% Like: 70% Dislike: 8% Dislike a lot: 1% High: 4% Medium: 83% Low: 13% n/a Science 8 435 (32/38) below Malaysia & Thailand (SS); above Philippines (SS) Boys scored 17 points higher (NS) High: 52% Medium: 47% Low: 0% High: 8% Medium: 73% Low: 19% 2003 Mathema tics 8 411 (34/45) below Malaysia (SS); above Philippines (SS) Girls scored 1 point higher (NS) High: 71% Medium: 28% Low: 1% High: 27% Medium: 59% Low: 15% High: 39% Medium: 45% Low: 16% Science 8 420 (36/45) below Malaysia (SS); above Philippines (SS) Boys scored 11 points higher (SS) High: 66% Medium: 33% Low: 1% (biology) High: 40% Medium: 53% Low: 7% (biology) 2007 Mathema 8 397 below Malaysia Girls scored High: 95% High: 28% High: 36% IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 11 tics (36/49) & Thailand (SS) 4 points higher (NS) Medium: 5% Low: 1% Medium: 58% Low: 14% Medium: 45% Low: 19% Science 8 427 (35/49) below Malaysia & Thailand (SS) Boys scored 2 points higher (NS) High: 88% Medium: 10% Low: 1% (biology) High: 41% Medium: 54% Low: 5% (biology) 2011 Mathema tics 8 386 (38/45) below Malaysia & Thailand (SS) Girls scored 13 points higher (SS) Value: 31% Somewhat Value: 61% Do Not Value: 8% Confident: 3% Somewhat Confident: 52% Not Confident: 45% Safe & Orderly: 37% Somewhat Safe & Orderly: 61% Not Safe & Orderly: 2% (reported by teachers) Science 8 406 (40/45) below Malaysia & Thailand (SS) Girls scored 7 points higher (SS) Value: 24% Somewhat Value: 62% Do Not Value: 14% (biology) Confident: 5% Somewhat Confident: 67% Not Confident: 29% (biology) 2015 Mathema tics 4 397 (44/49) n/a Girls scored 10 points higher (SS) n/a Very Confident: 23% Confident: 53% Not Confident: 24% Very Safe & Orderly: 89% Safe & Orderly: 11% Less than Safe & Orderly: 0% (reported by teachers) Science 4 397 (44/49) n/a Girls scored 8 points higher (NS) n/a Very Confident: 35% Confident: 46% Not Confident: 19% 1In 1995, TIMSS conducted assessments in grades 3-4, 7-8, and the final year of secondary education. Although Indonesia participated in Grades 4 and 8, the IEA did not release the data (Mullis et al., 1997). 2TIMSS tested only 8th grade students in 1999. NS: difference is not statistically significant SS: difference is statistically significant Participation in TIMSS Perhaps, the most important finding in Table 1 is that Indonesia has participated consistently in each round of TIMSS since 1995, providing a wealth of data for educational researchers. In contrast, Indonesia’s neighbors in Southeast Asia, especially the Philippines and Thailand, have not participated consistently. Malaysia did not participate in 1995, while has participated in eighth grade in each round since 1999. The Philippines has not participated in TIMSS since 2003, whereas Thailand did not participate in fourth or eighth grades in 2003. Additionally, Indonesia has participated in both eighth grade (1999-2011) and fourth grade (2015), which allows for examination of performance and school conditions across different grades and school configurations. On the other hand, several idiosyncrasies related to Indonesia’s performance on TIMSS complicate attempts to draw educational or policy lessons from the data. As I discussed above, despite the risks and costs associated with participating in international assessments, the benefits are far greater. Any failure to participate in an international assessment is bad news for a country seeking knowledge upon which to base educational decision making. See in this light, the failure to release Indonesia’s results in 1995 can be considered as a double disadvantage because the country incurred associated costs without receiving benefits of knowledge about the educational system or student performance. IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 12 Similarly, Indonesia’s lack of participation in the fourth grade TIMSS from 1999 to 2011 limits information available to educators and policy makers. Participation in fourth grade is important for a number of reasons. First, in many developing countries, student enrollment in school drops markedly in the upper primary and lower secondary grades. Since students who leave school during this period tend to be less advantaged in terms of their socio- economic status and academic achievement, students who remain in school at the secondary level are less representative of the population of children at the age than students in the fourth grade. In other words, the fourth grade data capture a more diverse and representative group of students. Second, by participating in both the fourth and the eighth grades, a country can roughly estimate whether a cohort of students has improved over time, by comparing the fourth grade achievement in one assessment to the eighth grade achievement four years later. This is not possible in Indonesia, given the country’s pattern of participation. Finally, Indonesia’s failure to participate in the eighth grade in 2015 limits the ability to draw lessons related to relative performance and trends over time. In the first case, most Southeast Asian countries that do participate consistently in TIMSS (especially Malaysia and Thailand) tend to participate at the eighth grade rather than the fourth grade level. As a result, we cannot compare Indonesia’s 2015 fourth grade performance with performance in Malaysia and Thailand, which participated in the eighth grade, while not the fourth grade. Second, Indonesia cannot compare the eighth grade achievement in 2011 to the eighth grade achievement in 2015. Whereas, the IEA’s 20 Years of TIMSS report describes trends across countries from 2011 to 2015, Indonesia is not included in these discussions because it did not participate in the same grades in these two years. This reason, in addition to the failure to release Indonesia’s data in 1995, explains why there is such little mention of Indonesia in the IEA’s 20 Years of TIMSS report. Student performance In each round of TIMSS, scores are standardized so that the average of all student scores is 500 and the standard deviation is 100. In 1999, Indonesia scored 403 in the eighth grade mathematics and 435 in the eighth grade science. In other words, Indonesia scored 97% of a standard deviation below the international average in mathematics and 65% of a standard deviation below the international average in science. 3 Of the 38 countries participating in 1999, Indonesia ranked the 34 th and the 32 nd in math and science respectively. In 2003, performance in math increased to 411, while the science score slipped to 420. The reverse occurred in 2007, with math performance decreased to 397 and science improved to 427. Scores in both math and science fell in 2011 to 386 and 406 respectively. In 2015, math performance increased to 397, while science performance decreased to 397. However, these data come from the fourth rather than the eighth grade, so it is difficult to compare them to earlier rounds of TIMSS. Student performance data indicated that; first, Indonesian children generally perform better in math than in science. Second, scores in both math and science have not increased 3 Although scores are not strictly comparable across years due to differences in participating countries, comparing standard deviations from the international mean provides a rough estimate of Indonesia’s relative performance over time. IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 13 steadily over time. Instead, scores in mathematics have fallen since 2003 (with the exception of the fourth grade scores in 2015), while science scores have fallen since 2007. Relative to other Southeastern Asian nations participating in TIMSS, Indonesia has scored consistently below Malaysia and Thailand, whereas consistently above the Philippines. However, given the patterns of participation by these countries in TIMSS (discussed above), we can only make straight comparisons for Malaysia in 1999, 2003, 2007, and 2011; for Thailand in 1999, 2007, and 2011; and for the Philippines in 1999 and 2003. Gender gaps in performance Among the countries participated in all rounds of TIMSS from 1995 to 2015, gender gaps between boys and girls generally declined, and the number of countries where girls outperformed boys in science and math increased. This has been the case in Indonesia. In 1999, the eighth grade boys outscored girls by 5 points in math and 17 points in science (differences were not statistically significant, or NS). By 2003, girls had closed the gap in math, scoring 1 point above boys (NS); in science, the gap between boys and girls dropped to 11 points (a statistically significant difference, or SS). In 2007, girls’ advantage in math increased to 4 points (NS) while boys’ advantage in science further declined to 2 points (NS). By 2011, girls had statistically significant advantages in both math (13 points) and science (7 points). In 2015, which tested fourth grade students, girls outscored boys by 10 points in math (SS) and 8 points in science (NS). Differences in student performance provide good news as they demonstrate that boys’ advantage over math and science has completely vanished. Unfortunately, a new gap favoring girls emerged during this period. Ideally, Indonesia will be able to achieve gender parity in student performance in the future. Students’ attitudes and self-confidence in math and science In each round of TIMSS, the IEA has asked students about how much they enjoy or value math and science. However, the exact question and response framework have changed over the years. In general, these questions demonstrate that Indonesian students place a high degree of value on both math and science. In 1999, 92% of students reported that they “liked a lot” or “liked” mathematics, while 97% of students placed a high value on science. In 2003, 71% of students placed a high value on math, while 66% of students placed a high value on science (biology). In 2007, these percentages jumped to 95% of students placed a high value on math and 88% of students placed a high value on science (biology). In 2011, the IEA changed the question to directly assess how much students valued math. Here, the percentages dropped to 31% of students said that they valued math, and 24% of students said that they valued science (biology). In 2015, TIMSS participated at the fourth grade; since the IEA did not ask the fourth grade students on how much they valued math and science, we cannot compare 2015 data to data from earlier assessments. Given the changing nature of questions related to students’ attitudes, it is difficult to discern trends over time. However, we can infer that in general, students like or value math more than science. Additionally, while students appear to like math and science a great deal, they do not place as much value on math and science as one might expect given their positive attitudes about these subjects. IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 14 An additional anomaly emerges when we examine students’ responses to questions about their self-confidence in mathematics. With a few exceptions, Indonesian students appear to have very little confidence in either mathematics or science. In 1999, only 4% of the eighth grade students reported having a high degree of confidence in math, compared to 8% of students in science. In 2003, these figures jumped to 27% of students in math and 40% of students in science, and remained stable in 2007 at 28% and 41% of students respectively. In 2011, the IEA changed the response scale to “confident,” “somewhat confident,” or “not confident.” In that year, only 3% of Indonesian eighth grade students reported being confident in math, compared to 5% in science. In 2015, the scale changed again, and the fourth grade students participated, rather than the eighth grade students. The good news is that 23% of Indonesian fourth grade students reported they were “very confident” in math and 53% reported they were confident; scores for science were even higher, with 35% of students reported they were “very confident”, and 46% of students reported they were “confident.” Again, changes in questions, response scales, and grades make discerning trends in confidence over time. However, it seems that for the most part, Indonesian students are more confident in science than math, and the fourth grade students are more confident than the eighth grade students in both math and science. School climate and safety One of the brightest spots for Indonesia’s TIMSS participation over time is a high degree of school safety reported by students. Although the IEA did not ask a related question in 1999, in 2003, 39% of eighth grade students reported a high degree of safety in school. This figure dipped slightly to 36% in 2007. In 2011 and 2015, the IEA asked this question of teachers rather than students. In 2011, 37% of teachers reported that their schools were safe and orderly; in 2015, 89% of the fourth grade students’ teachers reported that their schools were safe and orderly, the highest percentage across all countries participated in the 2015 fourth grade TIMSS. Like the other measures reported above, it is difficult to discern trends in school safety due to changes in the nature of questions and responses over time. However, it is clear that at the fourth grade level, teachers feel a high degree of safety and orderliness at the school. Further, at the eighth grade level, a large majority of the eighth grade students reports that their school has a high or medium degree of safety, while a small minority of students reports a low degree of feeling safe at school. Discussion Indonesia’s 20 years of participation holds both good and bad news for student performance and related research. On the positive side, Indonesia’s consistent participation in TIMSS has produced a great deal of rich comparative data to analyze key trends and policy questions over time. Although published research using these data may be extensive in Bahasa Indonesia, a review conducted for this study yielded only a handful of peer-reviewed published studies in English. On the negative side, Indonesia’s participation patterns, beginning with the failure to report data in 1995 and continuing through the decision to participate at the fourth grade level in 2015, have complicated efforts to compare performance over time within Indonesia and across Southeast Asia. IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 15 In terms of student performance, math and science scores have generally remained quite low relative to other countries participating in TIMSS, with a negative trend in both math and science since the early 2000s. Further, Indonesia has consistently scored below its neighbors Malaysia and Thailand, while above the Philippines. Since 1999, the gender gap favoring boys has been completely reversed, with the fourth grade girls outscored boys in both math and science in 2015. In examining students’ attitudes about math and science, we find that students place high value on both subjects (somewhat higher on math), whereas have low degrees of confidence in both subjects, with student confidence somewhat higher in science and in the fourth grade. Finally, both students and teachers report relatively high levels of school safety and orderliness, with the fourth grade teachers reported the highest level of safety and orderliness among participating countries in 2015. Implications, Limitations, and Future Research This study provides a very simple descriptive analysis of Indonesia’s performance on TIMSS over time, and I use only a handful of measures of educational performance and equity. Consequently, I cannot offer major implications for educational research or policy making in Indonesia. The main conclusion of this review is that Indonesia faces many challenges as it seeks to improve educational access and quality for the millions of children in its primary and secondary schooling systems. Educational performance in Indonesia has appeared to stagnate, and students’ confidence in math and science is low. However, students have positive attitudes about these subjects and appear to feel safe in school. Rather than providing definitive lessons and implications, this study points the way for future researchers wishing to explore key questions related to educational access, quality, and equity in Indonesia. To explore and answer these questions, however, researchers will need to apply both descriptive and multivariate analysis to the TIMSS data. Many questions await these analyses. For example, how does educational performance vary across Indonesia’s diverse regions, language groups, and socio- economic groups, and what are the determinants of gaps between groups? Additionally, what accounts for the reversal of Indonesia’s gender gap in math and science, from a gap favoring boys to a gap favoring girls? Has boys’ performance decreased, has girls’ performance increased, or have both occurred? Further, what has been the impact of important policy reforms, such as Indonesia’s 2005 teacher certification law, on the percentage of certified teachers, the distribution of certified teachers across regions and schools, and student performance? Finally, what are the determinants of students’ and teachers’ perceptions of being safe at school, and do students and teachers in different locations perceive safety differently? The TIMSS data provide a world of data to be explored. Meanwhile participation in TIMSS is not enough. Indonesian researchers must take the lead in using these data to answer the big questions that will lead to improved educational practice and policy in the world’s fourth largest country. IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 16 References Chudgar, A. & Luschei, T. F. (2009). National income, income inequality, and the importance of schools: A hierarchical cross-national comparison. American Educational Research Journal, 46(3), 626-658. Chudgar, A., & Luschei, T. F. (2014). International assessments. In D. J. Brewer & L. O. Picus (Eds.), Encyclopedia of Education Economics and Finance (pp. 416-418). Thousand Oaks, CA: Sage Publications. Chudgar, A., & Luschei, T. F. (2016). The untapped promise of secondary data sets in international and comparative education policy research. Education Policy Analysis Archives, 24, 113. Heyneman, S. P., & Loxley, W. A. (1983). The effect of primary-school quality on academic achievement across twenty-nine high-and low-income countries. American Journal of sociology, 88(6), 1162-1194. Martin, M. O., Mullis, I. V., Gonzalez, E. J., Gregory, K. D., Smith, T. A., Chrostowski, S. J., ... & O'Connor, K. M. (2000). TIMSS 1999 international science report: Findings from IEA's repeat of the third international science and science study at the eighth grade. Chestnut Hill, MA: International Study Centre, Lynch School of Education, Boston College. Martin, M.O., Mullis, I.V.S., & Foy, P. (with Olson, J.F., Erberber, E., Preuschoff, C., & Galia, J.). (2008). TIMSS 2007 International Science Report: Findings from IEA’s Trends in International Mathematics and Science Study at the Fourth and Eighth Grades. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Martin, M. O., Mullis, I. V. S., Foy, P., & Hooper, M. (2016). TIMSS 2015 International Results in Science. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: http://timssandpirls.bc.edu/timss2015/international-results/ Martin, M.O., Mullis, I.V.S., Foy, P., & Stanco, G.M. (2012). TIMSS 2011 International Results in Science. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Martin, M.O., Mullis, I.V.S., Gonzalez, E.J., & Chrostowski, S.J. (2004). Findings From IEA’s Trends in International Mathematics and Science Study at the Fourth and Eighth Grades. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Mullis, I. V., Martin, M. O., Beaton, A. E., Gonzalez, E. J., Kelly, D. L., & Smith, T. A. (1997). Mathematics Achievement in the Primary School Years: IEA's Third International Mathematics and Science Study (TIMSS). TIMSS International Study Center, Center for the Study of Testing, Evaluation, and Educational Policy, Campion Hall, School of Education, Boston College, Chestnut Hill, MA 02167; world wide web: http://wwwcsteep. bc. edu/timss. Mullis, I.V.S., Martin, M.O., & Foy, P. (with Olson, J.F., Preuschoff, C., Erberber, E., Arora, A., & Galia, J.). (2008). TIMSS 2007 International Mathematics Report: Findings from IEA’s Trends in International Mathematics and Science Study at the Fourth and Eighth Grades. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Mullis, I.V.S., Martin, M.O., Foy, P., & Arora, A. (2012). TIMSS 2011 International Results in Mathematics. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Mullis, I. V. S., Martin, M. O., Foy, P., & Hooper, M. (2016). TIMSS 2015 International Results in Mathematics. Retrieved from Boston College, TIMSS & PIRLS International Study Center website: http://timssandpirls.bc.edu/timss2015/international-results/ IRJE | Vol. 1 | No. 1| Year 2017 |ISSN: 2580-5711 17 Mullis, I. V., Martin, M. O., Gonzalez, E. J., & International Association for the Evaluation of Educational Achievement. (2000). TIMSS 1999 international mathematics report: Findings from IEA's repeat of the Third International Mathematics and Science Study at the eighth grade. International Study Center. Mullis, I.V.S., Martin, M.O., Gonzalez, E.J., & Chrostowski, S.J. (2004). Findings from IEA’s Trends in International Mathematics and Science Study at the fourth and eighth grades. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Mullis, I. V. S., Martin, M. O., Loveless, T. (2016). 20 years of TIMSS: International trends in mathematics and science achievement, curriculum, and instruction: Boston: TIMSS International Study Center. Biographical note THOMAS LUSCHEI is an associate professor in CGU’s School of Educational Studies, where he teaches courses on international comparative education, urban education in the United States, and teacher quality and teacher policy. His research interests include international and comparative education, the economics of education, teacher labor markets and teacher quality, bilingual education policy and practice, and the global applications of Colombia’s Escuela Nueva rural school improvement model. The primary focus of his research is the impact and availability of educational resources—particularly high-quality teachers—among economically disadvantaged children. He has conducted related research in Africa, Asia, and North, Central, and South America. In 2013, he served as a Fulbright Visiting Scholar at the Centro de Investigación y Formación en Educación at the Universidad de los Andes, Bogotá, Colombia. In 2016, he was a Fulbright Specialist at the Universidad de San Andrés, Buenos Aires, Argentina. Luschei is the author (with Amita Chudgar) of Teacher Distribution in Developing Countries: Teachers of Marginalized Students in India, Mexico, and Tanzania (2016, Palgrave Macmillan). His research has appeared in numerous scholarly journals, including the American Educational Research Journal, American Journal of Education, Comparative Education Review, International Journal of Educational Development, Education Policy Analysis Archives, International Review of Education, Phi Delta Kappan, Teachers College Record, and Prospects: UNESCO’s Comparative Journal of Curriculum, Learning and Assessment. He serves on the editorial boards of the journals Educational Researcher, Evaluation and Program Planning, Interaction Design and Architecture, Forum for International Research in Education, Journal of Teacher Education, Panorama, and Prospects. He has been invited to present his research in Argentina, Colombia, India, Indonesia, Mexico, and South Korea. From 2011 to 2014, he served on the board of directors of the Comparative and International Education Society. Prior to pursuing graduate studies, he worked as a bilingual elementary, high school, and adult education teacher in the Los Angeles Unified School District, where he taught students ranging in age from six to 84.