Review of Education by the numbers and the making of society Oromaner, M. (2019. May 1). Review of Education by the numbers and the making of society: The expertise of international assessments by S. Lindblad, D. Pettersson & T. S. Popkewitz (Eds.). Education Review, 26. http://dx.doi.org/10.14507/er.v26.2436 May 1, 2019 ISSN 1094-5296 Lindblad, S., Pettersson, D., & Popkewitz, T. (Eds.). (2018). Education by the numbers and the making of society: The expertise of international assessments. New York, NY: Routledge. Pp. 247 ISBN: 9781138295834 Reviewed by Mark Oromaner Independent Scholar United States More than 6o years ago, the Russian-American sociologist Pitirim Sorokin argued that the misplaced stress on testing and quantitative analysis in the psychological and social sciences reflects a phase that “can be properly called the age of quantophrenia and numerology” (Sorokin, 1956, p. 103; italics in the original). He goes on to argue that this “disease” manifests itself in every region of sociology, psychology, psychiatry, and anthropology. Given the increasing prevalence and acceptance of testing and quantitative analysis in all aspects of education, we can add education to Sorokin’s list of fields that are affected by this “disease.” In the words of the editors of Education by the numbers and the making of society, “What is regarded as valid knowledge on and in education is to a very high extent to know education by the numbers” (p. viii). Education by the numbers (EbN) is not limited to assessments within schools, districts, or even nations. As the subtitle of the book states, EbN explores the expertise of international assessments. The field of Education Review 2 comparative empirical education studies is often traced to the early 19th-century French scholar Marc-Antoine Jullien, who was influenced by positivism and the study of social physics (named sociology by Auguste Comte). During the 20th century, comparative education data were collected and disseminated by international organizations such as UNESCO, the World Bank, and the Organization for Economic Co-operation and Development (OECD). Today, the OECD’s Programme for International Student Assessment (PISA) is one of the most widely used comparative EbN, and is the primary instrument analyzed in the book. This volume presents an extremely timely analysis of EbN; however, it is more than merely timely. The book offers a groundbreaking application of the social construction of social science knowledge to education statistics, research, assessment, and policymaking. Although some sections will be of more interest to certain readers than others, anyone interested in educational assessments will find much of value in the book. In addition to the view that EbN is always a socially produced process, throughout the book the authors remind us that numbers not only describe present and past categories, but also describe a desired future. International comparisons serve as “rationales” for policy implementation. For instance, if economic development is based on proficiency in STEM subjects, policy makers may decide to invest resources in these disciplines and in students who are high achievers in those disciplines. Education by the numbers contains 15 chapters divided into an introduction and four sections. Although the introduction is co- authored by the three editors, none of them participated in the writing of 12 of the 14 remaining articles. All articles are well documented, thoughtful, and written by knowledgeable scholars and researchers. In addition, they do not require more than an elementary understanding of the use of numbers. Section I comprises four articles on “Numbers: A History of a Style of Reasoning.” The authors provide a historical framework and case studies to understand the relationship between the state and statistics. In his informative essay “Politics by the Numbers,” Theodore M. Porter points out that although statistics may have served as a descriptive science that enabled leaders to understand what was happening throughout their jurisdictions (e.g., census, economic development), “Statistics has become a fundamental instrument for creating and shaping public visibility” (p. 29). Hans Krause Hansen and Anne Vestergaard present a cautionary tale for anyone who uncritically accepts quantitative data as a basis of remediation of international social issues. They demonstrate that, depending on how lists are created, countries most responsible for tax havens differ. A list of the OCED indicates that mostly small and island states are responsible for the existence of tax havens. On the other hand, the Financial Secrecy Index suggests that many of the leading countries, in terms of secrecy of financial transactions, are among the world’s largest and wealthiest countries. The data and comparisons do not speak for themselves; one must understand how they were constructed. Section II comprises four articles that focus more closely on the issues surrounding the problematics of educational assessment. In their article on the development of OECD indicators, Regula Burgi and Daniel Trohler argue that the “comparative turn” in educational assessment was not a new phenomenon associated with post-Cold War neoliberalism and globalization. Instead, it was a stage in the development of comparative education leading from forecasting (quantitative description) to planning (development and growth to reach bench marks) to management (changing all levels of educational systems). The measurement of Review of Education by the Numbers by M. Oromaner 3 performance of these managed changes meant that qualitative aspects had to be translated into numbers, and “… those numbers were not to be descriptive but were normative, carriers of meaning, indicating how good or bad an education system is performing” (p. 87). Other authors emphasize the limitations of programs such as PISA for making international comparisons. Radhika Gorur points to the limited decision-making process involved in accepting the standardization of concepts required for such comparisons and writes about “the illusion of comparability.” Do terms such as course, school year, teacher qualification, and literacy have the same meaning across nations? What does literacy rate mean if one nation defines it in terms of the ability to sign one’s name or to read a simple paragraph and another to the ability to read and write a short statement? Given the limitations of PISA for the production of evidence-based policymaking, Luis Miguel Carvalho views it as a “form of persuasion through knowledge…” (p. 111). I suggest that “socially constructed knowledge” is a better term. Through reports, videos, databases, and information technology (IT), the OECD disseminates PISA results to national and international governments, think tanks, consortia, and local experts. At times, databases are forwarded to targeted audiences for use in do-it-yourself analyses. For an excellent collection of articles concerning the ways in which mass media, specialized media, IT, and think tanks use and are used by educational assessment programs see Wubbana, Ford & Porfilio (2016). In what may be the most valuable article for opinion makers, education researchers, and policy makers, David Berliner explores the impact of the common sense, but often forgotten, view that PISA, “… is merely another Standardized Achievement Test” (p. 127). For instance, he elaborates on a point cited above that it is difficult to believe that “item equivalence” can be assured across 65 nations that participated in the 2012 PISA program. Problems encountered in intranational studies (e.g., regional languages and dialects, subcultures, prior exposure, and context) are exacerbated in international studies. Small differences in national raw scores due to differences cited above become magnified when sophisticated statistical models are employed. Thus, for technical reasons, identical or similar raw scores may result in different scaled scores and rank positions among and between nations. Later in the volume, authors examine the process of the dissemination of OCED education assessment data in order to influence various audiences. In the first article, authors examine how the results of the 2015 PISA tests were received over time and according to geography on Twitter. In one article, authors move from PISA to the Trends in International Mathematics and Science Studies (TIMSS) program. They employ methodologies and concepts from the sociology of science, information science, and bibliometrics (e.g., citation analysis) to identify the most highly cited articles using or discussing TIMSS. From these, they were able to identify authors, journals, institutions, and countries that utilized (cited) these papers. In addition, through co-citation and cluster analyses the authors were able to visualize the intellectual organization of the utilization of TIMSS data (i.e., psychology, educational research and subject specific research) (p. 174). In another article, Sotiria Grek explores the ways in which the OECD has become a powerhouse in the dialectic between the boundaries of national European education policies and the transnational policies given legitimacy by the international assessments it produces. A strength of this collection is that it focuses on the role of non-government organizations (e.g., OECD, think tanks) in educational assessment and policy making. However, the current nationalist policies in the United States (i.e., Trumpism) and in Europe, (i.e., Brexit and Orban in Hungary) suggest Education Review 4 that a backlash against international assessments is possible. Finally, the book offers a broader look at the social construction of statistical reasoning and its consequences. Various articles make a historical point concerning our perception of reality. Prior to the 19th century, numbers were about individuals; since then statistical reasoning has made it possible to think about larger aggregates. This leads to a classification of individuals into abstract categories (e.g., above or below average, deviant, at-risk, immigrant, and the like.). These are not neutral categories, but indicate who is to be included, who is to be excluded, and who is in need of intervention or special attention. Thus, according to Popkewitz and Lindblad, “… governing is exercised less through brute force and more through the system of reason that fabricate kinds of people and biographies” (p. 216). Education by the numbers raises questions concerning the interaction between science and society in general and takes a major step toward a response to the fundamental question: “What are the implications of such a comparativistic turn in educational policy and in the making of education and schooling?” (p. 18). Although the wide-ranging content from specifics of assessment programs to the sociology of knowledge and the history of statistics is admirable, it is not realistic to believe that all readers will be equally conversant or even concerned with some of these topics. As an aside, a strength of this collection is that authors often cross-reference and therefore enhance its integration. Lindblad, Pettersson, & Popkewitz recognize that the book moves in two directions at the same time: (1) case studies of comparative education assessments, and (2) historical and social/political changes of the relationship between science and society. In anticipation of the audience for this review, I have emphasized the former. However, I recommend that graduate students, scholars, researchers and education policymakers step outside of their intellectual and interest zones and silos, and read the wisdom in EbN. References Sorokin, P. (1956). Fads & foibles in modern sociology and related sciences. Chicago: Henry Regnery Company. Wubbana, Z. C., Ford, D. R., & Porfilio, B. J. (Eds.), (2016). News media and the neoliberal privatization of education. Charlotte, NC: Information Age Publishing. About the Reviewer Mark Oromaner, a sociologist, is an independent scholar. He retired after having spent more than 35 years in higher education institutions as a faculty member, administrator, and researcher. His major research interests and publications focus on the sociology and history of higher education and on the social studies of the social sciences. His research has appeared in journals such as The American Sociologist, British Journal of Sociology, Research in Higher Education, Scientometrics, Philosophy of the Social Sciences, Social Studies of Science, and Knowledge: Creation, Diffusion, Utilization. Review of Education by the Numbers by M. Oromaner 5 Education Review/Reseñas Educativas/Resenhas Educativas is supported by the edXchange initiative’s Scholarly Communications Group at the Mary Lou Fulton Teachers College, Arizona State University. Copyright is retained by the first or sole author, who grants right of first publication to the Education Review. Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Review, it is distributed for non-commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or Education Review. Education Review is published by the Scholarly Communications Group of the Mary Lou Fulton Teachers College, Arizona State University. Disclaimer: The views or opinions presented in book reviews are solely those of the author(s) and do not necessarily represent those of Education Review. Connect with Education Review on Facebook (https://www.facebook.com/pages/Education- Review/178358222192644) and on Twitter @EducReview https://www.facebook.com/pages/Education-Review/178358222192644 https://www.facebook.com/pages/Education-Review/178358222192644