Microsoft Word - v9n4.doc Analyzing Education Productivity: An Essay Review Ibrahim Duyar University of Arkansas at Little Rock Citation: Duyar, Ibrahim. (2006, August 26). Analyzing education productivity. Education Review, 9(4). Retrieved [date] from http://edrev.asu.edu/essays/v9n4/. Subotnik, Rena and Walberg, Herbert. (Eds.) (2006). The Scientific Basis of Educational Productivity. Greenwich, CT: Information Age Publishing, Inc. pp. viii + 300 $34.95 ISBN 1-59311-449-4 The concept of productivity was born in the field of economics to minimize the costs and maximize the outputs. In its simplest form, productivity can be defined as achieving the maximum output of a process with the use of minimum inputs. Organizations are in continuous search of the best technology and methods of using minimum inputs to produce maximum outputs to become competitive and survive in the market. Productivity can be applied to the field of education the same way in which economists analyze the relationships between inputs and outputs (Duyar, McNeal, & Kara, 2006). Although becoming competitive to survive may not be their main motivation, public education institutions are also expected to be productive to minimize costs and maximize the utilization of resources to meet increased and diversified needs, as well as to become accountable to the public for the expenditure of resources. In this sense educational productivity can be defined as the efficient production of educational outcomes (Rolle, 2004). Research in educational productivity is traditionally conducted by economists, who relate inputs, such as expenditures per student, to outcomes, such as student achievement, through the utilization of production function analysis. Production function research has attempted to estimate relationships between the cost of selected schooling inputs and educational outcomes, controlling the influence of various background features (Monk, 1992). Although rich in history, this line of research in education has not been consistent or conclusive. It Education Review Vol. 9 No. 4 http://edrev.asu.edu 2 has rarely produced significant results in identifying educational inputs, their usage in educational production functions, or linkages between inputs and student learning outcomes. This line of research has not been able to define an educational productivity function adequately, thus, it has yielded little to guide education policy makers. Current productivity issues in education are no less pressing than when production function research started fifty years ago. The American public has noticed a decline in the productivity of the education system and has demanded improvement in educational productivity. In fact, as Hoxby (2004) asserts, the productivity of American public schools fell by approximately half from 1970 to 2000. Policymakers are now more in need of guidance in improving the productivity of the education system than ever before. In response, educational researchers have continued over the years to address unanswered education productivity questions by applying more advanced techniques and methods to traditional frameworks and developing alternative frameworks of education productivity. Toward this end, the Scientific Basis of Educational Productivity by Rena Subotnik and Helbert Walberg exemplifies recent significant attempts to realize the ultimate goal of educational productivity, which is to improve the quality and quantity of educational opportunities for children. Their book is founded on the idea that education research can, and must, be rigorous to contribute substantially to education reform and improvement of American students’ achievement. Although a variety of scientific approaches to education productivity research are represented, the book emphasizes the special credibility of randomized experimental methods. Thus, the book calls attention to the scarcity of experimental research despite universal acknowledgement that controlled experiments provide the best warrant for causal conclusions. Experimental research responds to the urgent needs of educators and policymakers in search of definitive findings about what effectively and efficiently raises achievement and contributes to the success of their students. This review is organized in four sections. Following this introduction, the issues of research in education productivity are analyzed within the context of the education production function approach. These issues include: (a) conceptual misfit; (b) technological difficulties; (c) disregarding process of instruction and learning; and (d) methodological misfit. Alternative approaches to educational productivity are examined in the third section. These approaches include: (a) principal-agent theory; (b) quadriform analysis; (c) data envelopment analysis; and (d) stochastic frontier analysis. The final section reviews the book both contextually and substantively. While the contextual overview discusses the main focus and contribution of individual chapters in the book, the substantive overview examines the overall contribution of the book to the education productivity literature. Prospects for education productivity research in light of the ideas presented by Subotnik and Walberg conclude the review. Duyar: Analyzing education productivity: An essay review 3 Analysis of the Productivity Problem in Education: Production Function Research Improving education productivity always appears to be on the policy and research agenda. Historical monetary figures show that the American public has been increasingly generous to public education. Public education today is a $450 billion a year sector of the U.S. economy; each school day costs $2.5 billion. Even after adjusting for inflation, education spending has increased significantly during the last five decades (NCES, 2005). Unfortunately, the productivity dilemma surfaces when one examines student learning levels in core subjects as measured by standardized tests. Student achievement levels have not increased at the same rate as resources (Odden, 1992). To make the matter even worse, the gap between increasing inputs in the face of stagnant or inert outcomes has been constantly widening; the result is a steady decline in educational productivity over time. For instance, the productivity of American public schools fell by approximately half from 1970 to 2000 when test scores and the per-pupil spending are compared. The productivity of American public schools is lower than that of many other industrialized countries, including the other English-speaking ones (Hoxby, 2004). Because of this downward trend in educational indicators in the last fifty years, the American public has been increasingly aware that their children need to learn more with given amount of resources and in a given interval of time and our schools need to be more productive. Why has public education been experiencing declining productivity? Why has the research on education productivity failed to guide educators and policymakers on what effectively and efficiently raises achievement of students? What went wrong with research on education productivity? Why has education research failed to produce consistent, definitive, and significant findings to increase education productivity? To find the answers to these questions, one needs to examine the research on education productivity on theoretical and practical grounds. Research on educational productivity is traditionally conducted by economists who apply a production function method to the production of student learning outcomes. The production function approach to educational productivity relates inputs, such as expenditures per pupil, to outcomes, such as student achievement (Odden, 1992). It started with attempts to estimate relationships between the supply of selected inputs and educational outcomes, controlling for the influence of various background features (Monk, 1992). The selected inputs included various attributes of students, teachers, and school organizations. Student ability and family socio-economic status were among the often controlled background features. Despite these individual inputs, many of the production function studies have tended to use cost as an aggregate measure of schooling inputs (Rolle, 2004). After all the conceptual and technological advancement, there is consensus that the existing education production function research has been largely unsuccessful at revealing a production function that dependably contributes to enhanced learning gains of students. This section examines the main reasons for the failure of education production analysis. The first reason for inconsistent results is the conceptual misfit of the production function model. The conceptual inadequacy of the underlying productivity model stems from the assumptions of this model and the political nature of the education system. The second reason includes technological difficulties in identifying educational inputs and outputs. The third reason is the tendency of education production researchers to overlook the classroom level Education Review Vol. 9 No. 4 http://edrev.asu.edu 4 and to disregard the process of instruction and learning. A review of literature shows that a significant number of education productivity studies were conducted at the school district, regional, and state level of analysis. Finally, the methodological misfit uncovers the realities of the education process that have prevented education production research to produce consistent and conclusive findings. Conceptual misfit. The production function is a logical input-outcome framework for the production of goods and services in the free-market economy. Its application to public education has fallen short in explaining the dynamics of public education production. The public service nature of the education system entails intricacies that defy the simple one-to- one relations embodied in the production function analysis and cause imperfections in the public education market (Duyar, 1996). Therefore, as Monk (1992) stated, despite growing econometric sophistication, education production studies “remain fundamentally primitive black-box formulations where analysts have made little progress toward modeling what makes education distinct from other types of production more typically studied using production function techniques” (p. 309). The issues in the applicability of free-market conceptual frameworks to bureaucratic public goods and services markets have not been resolved yet. The recent applications of market-based applications to public schooling include charter schools, outsourcing, and educational management organizations, which raise significant issues that are yet to be studied (Duyar, 2004). Without reconciliation of the differences between free-markets and public education markets, the applications of a framework appropriate to one to the other may result in complications. Of course, such reconciliation can only be possible when production function research is able to model the dynamic and changeable nature of the education production process or when the alternative approaches to education productivity that respond to the limitations of the education production function approach are applied. To understand the conceptual misfit, one needs to look into the assumptions of the production function research. Application of the cost minimization assumption of the production function has created misfits with education production. In this regard, the production function studies assume that public school organizations and their administrators act as cost-minimizing agencies, similar to their private business counterparts. Research on the behavior of public school agencies and their administrators, however, has shown that these agencies and their administrators are much more likely to be revenue maximizers (Barnett, 1994; Rolle, 2003). The inapt assumption of cost minimization and the failure to model the changeable nature of the education production process are serious limitations on the conceptual fit between education production function models and the realities of the public school system. Difficulties in identifying inputs and outcomes. Education production function research has experienced special difficulty in identifying education inputs. Earlier production function studies used cost or expenditures as the crude proxy for inputs. As discussed above, this approximation yielded complications when applied to public education. For instance, this model excluded nonpurchased inputs from the production functional model. Thus, these studies overlooked the effects of some of the important inputs on education outcomes. Numerous studies have confirmed the significant influence of nonpurchased inputs, such as peer effects (Dills, 2005; Harker & Tymmes, 2004), parental Duyar: Analyzing education productivity: An essay review 5 effects (Hill & Taylor, 2004), and school level effects (Zvoch & Stevens, , 2006; Bishop & Wößmann, 2004) on educational outcomes. Education production function research has used a deductively driven model in identifying and selecting inputs. As Monk (1992) stated, even the modern studies in production function research tend to justify their selection of variables with references to previous studies that looked at similar studies. The derivative nature of these studies might limit the improvement of production model and its explanatory power. Difficulties in identifying purchased inputs, nonpurchased inputs, and relationships between groups of inputs continue despite the improvements in econometric production function techniques. Untangling the effects of the configuration of inputs has attracted a considerable amount of research; but as Cooper and his colleagues (1994) warned, true productivity relations between educational inputs and student outcomes are still are unknown. Disagreement on outcome variables in education production function analysis has been significantly less serious than disagreement on identifying inputs. Some studies have used test score performance of students and others have used labor market success of the graduates. Recent production function studies (Ma &Wang, 2004; Wang & Staver, 2001; Young, Reynolds, & Walberg, 1996) examined both student achievement and labor market success, with the latter of the two being the ultimate outcome variable. Disregard of the process of instruction and learning. Early production function research focused on the relationship between inputs and outcomes without much attention to the production process. These studies ignored the classroom as the unit of analysis and focused at the level of districts or states. Disconcerting patterns of inconsistent and often insignificant results forced researchers to study the classroom to estimate the education function in the 1970s. These researchers emphasized the importance of the micro-approach, complete with disaggregated data and a focus on decentralized levels of decision making. However, judging from the review of literature that emerged in the next three decades, the attempts to discover the regularities of the education production process at the classroom level have made little impact on policy making. One of the reasons for the weak impact of this research on policy decisions is the missed opportunities to make connections across the disciplines of education and economics. There has been an impressive amount of intellectual energy devoted to the use of economic models to study schooling and classroom processes. There has also been a parallel but largely unconnected development within the field of education via micro-approaches to educational productivity. While the work of economists has been highly technical and has exercised little influence on policy, the work of educators has been less than comprehensive because of the relative lack of sophistication in economics. Both sides have showed little effort to collaborate and bridge the gap between the two disciplines. Unfortunately, Subotnik and Walberg's attempt to bridge this gap is somewhat weak. Educators’ views dominate The Scientific Basis of Educational Productivity, which excluded the only economist’s view that appeared on the original prospectus for the book. Education productivity research may make greater progress if collaboration between educators and economists is established. Education Review Vol. 9 No. 4 http://edrev.asu.edu 6 Methodological misfit. Finally, methodological misfit has prevented education production research from coming up with consistent and conclusive findings. The early production research methodology relied heavily on survey data, correlational research designs, and multivariate statistical analyses. As stated above, rather than pursue a derivative strategy to untangle the dynamics of education production, there is a need to examine the dynamics of the instruction and learning process through the use of an internal logic and by looking for evidence of success to see if the ideas introduced work in practice. There is a growing interest in the use of randomly controlled experimental designs in the study of the education production process. The momentum that calls for more experimental studies in education productivity research has not slowed despite the development of new econometric techniques of estimation. These new techniques, such as hierarchical models, use random coefficient models that permit researchers to estimate differences across units in the nature of phenomena occurring within units (Young, Reynolds, & Walberg, 1999). These new econometric models deal with the nested nature of decision making and the reciprocal nature of the relationships that exist across levels, and therefore, should be viewed differently from their predecessors (Monk, 1992). Moreover, there has also been econometric progress in dealing with the dimension of the cost of education production. These advanced techniques are used to revisit the old notion of “does money matter” and to test the effects of traditional aggregate expenditure input levels on student performance. These studies have yielded positive results and have showed that there are factors controllable by school systems that increase educational productivity. For instance, Hedges and his colleagues (1994) found that an approximate 10% or $500 increase in spending per pupil significantly increases student achievement. Unfortunately, many retain a degree of skepticism towards these positive findings and do not see them as conceptual progress in educational productivity research. This line of research has to prove that it is different from the other positive studies that can be deliberately selected (or "cherry picked" one might say) from the entire body of inconsistent and largely statistically insignificant results. Subotnik and Walberg call for more randomly controlled experimental design studies in education productivity research. Almost all the authors of this book emphasize the importance of these designs in education productivity research. It should be noted, however, that experimental design at the classroom level poses limitations for economists who have traditionally tended to use large scale survey data. As Cook (2002) states, educators have also been reluctant to run experimental studies despite the universal understanding that experiments provide the best justification for causal conclusions. Educators’ reluctance to conduct experimental research, mostly due to the high costs associated with this type of research, is also evident in one of the chapters. Pang and Kamil (2006) reported only 39 studies as experimental and quasi-experimental out of 306 studies they reviewed for the analysis of research on the reading strategies taught to preservice and inservice teachers. Experimental studies offer new perspectives on the discovery of improvement of education productivity. However, there are at least two caveats that may limit the application of these Duyar: Analyzing education productivity: An essay review 7 studies to policy and reform efforts. As Subotnik and Walberg state, even the experimental studies are no guarantee of consistent, noncontroversial findings that translate straightforwardly into education policy and school reform. These studies may not necessarily escape the presence of inconsistencies. In fact, the early positive findings of demonstration programs, such as Success for All (Walberg, 2006), Accelerated Schools (Viadero, 2005), and the Tennessee class size reduction program STAR (Johnson, 2000), were not consistently confirmed by subsequent studies. Best practice demonstration programs searching the regularities of the production function are criticized for the risk of increasing the cost of pursuing what amounts to a hit-or-miss strategy for discerning regularities in education production (Monk, 1992). The second caveat is the ignorance of the use of experimental design by researchers. Despite the universal acknowledgement that the experiments provide the best justification for causal conclusions, such studies are scarce. First, economists were blamed for not using experimental design because of their weak knowledge on the education production process; now educators are behind in running experimental studies. As Cook (2001) stated, experimental studies were mostly done by groups outside of the community of educational evaluators working in schools and colleges of education. The results of these studies are usually retained at the institutions requiring the study, thus, limiting their influence on policy. Alternative Approaches to Education Productivity Despite all technical advancement, currently there is not a single identified education production function. Researchers continue attacking the education productivity problem by improving the production function framework and developing alternative frameworks. Continued attempts at improving the education production function framework gainsay the possibility that there is no such function, and assume that the inconsistencies are the result of technical problems. The attempt to improve the production function method includes the addition of features of the education process, and thus, corrects the error of over reliance on a simplistic input-outcome model of education production. Micro-approaches to education productivity research represent this line of inquiry. Education productivity research still yields studies attempting to defy the technical problems of the production framework by applying advanced econometric techniques to discover the dynamics of education production. As discussed above, the positive findings of these contemporary studies are faced with a great deal of skepticism and are forced to prove that they are not merely cases selected from an inconsistent and largely insignificant corpus of findings. Since we neither prove nor disprove whether there is in fact an education production function, some alternative frameworks offer a “quasi-production function” to best describe the underlying reality (Monk, 1992). Others, however, challenge the normative economic basis of education production framework and propose “public choice economic frameworks.” Public choice economic frameworks apply profit-maximizing analytical assumptions and focus on economic, organizational, and political incentives influencing the behavior of individuals and groups within the public education system (Rolle, 2004). Public choice economic approaches primarily acknowledge the sociopolitical environment in which both organizations and individuals struggle for legitimacy and the capacity to Education Review Vol. 9 No. 4 http://edrev.asu.edu 8 distribute scarce resources. Within this highly volatile environment, the actors are in a continuous search of maximum profits. As a result of this conflict of reality, values, and preferences, the ability to negotiate and compromise becomes an important asset for all actors. The resulting compromises generate a multiplicity of objectives that emerge as political, organizational, and personal goals. Therefore, when acknowledging these nonmarket influences on education productivity, examining alternative measures of efficiency is needed. This review briefly examines the four main public choice economic approaches to education productivity. They are (a) principal-agent theory, (b) modified quadriform analysis, (c) data envelopment analysis, and (d) stochastic frontier analysis. Principal-agent theory. Principal-agent theory is one of the early public choice economic approaches. It is extensively used and popularized by economists in explaining the behavior of a profit-maximizing firm. This theory suggests that a decentralized approach to management should be used when there is uncertainty in linking behaviors to outcomes, where objectives are divergent, and when there is information asymmetry between system leaders (the principal) and the service providers (the agent) (Kara, Duyar, Christy, &McNeil, 2006). Quadriform analysis. Quadriform analysis is an abstract tool devised to allow two- dimensional relations to be viewed graphically. Typically, student outcomes are measured along the vertical axis and expenditures are measured across the horizontal axis. In a two- by-two environment, education organizations may operate in one of the four quadriform: efficient, effective, ineffective, and inefficient. Quadrant 1 includes the efficient schools and districts. These educational organizations generate higher than expected expenditures. Quadrant 2 involves effective schools and districts which generate higher than expected educational outcomes using higher than expected expenditures. Quadrant 3 comprises ineffective schools and districts that generate lower than expected educational outcomes using lower than expected expenditures. Finally, Quadrant 4 consists of inefficient schools which generate lower than expected educational outcomes with higher than expected expenditures. Improved quadriform analysis is used to examine expenditure-output relations quantitatively and measure differential levels of economic efficiency among school districts (Anderson, 1999; Rolle 2003). Data envelopment analysis. Data envelopment analysis is used to evaluate the level of efficiency present in an organization relative to the best performing organizations in the investigated sample (Worthington, 2001). The focal point of data envelopment analysis is to determine statistically the best-performing organization. If the statistically determined best-performing organization has higher levels of economic productivity than the remaining organizations, the residual organizations are labeled as inefficient producers in relative comparison to the best-performing organization in its comparison group. Data envelopment analysis can be used to statistically identify the maximum combinations of outputs (production efficiency frontier) that can be produced for a combination of inputs. The performance efficiency ratio of each school organization is calculated by comparing the school’s actual performance value with its statistically determined efficient value on the production efficiency frontier. Linear programming methods are used to identify efficiency levels of school in a school district by using multidimensional models. (Rolle, 2004) Duyar: Analyzing education productivity: An essay review 9 Stochastic frontier analyses. Stochastic frontier analyses focuses on determining statistically the best-performing organization as with the modified quadriform analysis and data envelopment analysis. Similar to data envelopment analysis, stochastic frontier analysis also uses an efficiency frontier estimate to measure the levels of relative efficiency. Depending on the availability of the data, stochastic frontier analysis can allow for the district measurements of allocative and technical efficiency (Rolle, 2004). Acknowledging the political nature of the education production process and acknowledging the nonmarket influences on educational productivity are some of the main differences between alternative models and production function models of education productivity. Alternative approaches to education productivity have also allowed for examination of the areas which need to be operationalized and explored. For instance, data envelopment and stochastic frontier analysis can be used in expanding the traditional two-stage production function relations into multi-stage models that more accurately portray the educational process. Hierarchical and nonlinear statistical models and relations that more accurately represent the educational process for all students and student subgroups can be examined. Furthermore, exploring theoretical and statistical relations that more accurately represent the educational process for all students and subgroups of students using multiple output regression models is also possible with these alternative models. Despite the advantages, the alternative models to educational productivity have similar disadvantages that the production function research is experiencing. These models also use cost as the aggregate input variable and they mostly focus on the economic efficiency in education. Technical efficiency and the dynamics of resource allocation decision making also need to be taken into account. Although these models claim to address the limitations of the production function regarding the political nature of the public education process, the earlier alternative studies also have overlooked the process of instruction and learning. Rather, they focused on school or district as the unit of analysis, as did their precedents. The student or student subgroup analysis are yet to be studied through these alternative models. More importantly, the early alternative models also lack the use of experimental design. Subotnik and Walberg’s study makes another plea to fill this gap in studying the educational productivity in scientific grounds. The Scientific Basis of Educational Productivity The Scientific Basis of Educational Productivity emerged from a conference co-sponsored by the Laboratory for Student Success and the American Psychological Association. The main idea of the book and its companion conference is that education research can, and must be, more rigorous to substantially contribute to education reform and the improvement of performance of American students. Although a variety of scientific research approaches are represented, the book emphasizes the significance of experimental and mixed method studies converging on the same policy and practice relevant results. Subotnik and Walberg view the neglect of rigorous experiments as the main reason for the current shaky scientific grounds underlying educational research. Research on K-12 education lacks a strong foundation in causal research, particularly discipline-based, control group experiments and large-scale, well-controlled, statistical studies. In that sense, The Scientific Basis of Educational Productivity addresses the methodological misfit of current production function research and Education Review Vol. 9 No. 4 http://edrev.asu.edu 10 may be considered as a work that supports alternative approaches to education productivity analysis. The significance and timeliness of The Scientific Basis of Educational Productivity is evident. It is an attempt to respond to the growing frustration with the inconclusive policy implications associated with education productivity research. As evidenced by the federal No Child Left Behind Act and more scrupulous testing and accountability systems in the states, rigorous methods to assess the effectiveness of educational interventions are particularly needed for public school reform. The American public, policymakers, and educators greatly desire findings about what effectively and efficiently raises student achievement and raises the productivity of public education system. In this regard, The Scientific Basis of Educational Productivity promotes scientifically based policy and practice designed to ensure academic and life success for students in our public schools. The book is designed to identify evidence- based practices and effective programs for the policy makers through the use of rigorous scientific educational research methods. The purpose of this book was to find distinguished authorities with different views who could shed light on this aspect of education. The book does not only target scholars in a variety of academic disciplines but also research consumers, including educators, policymakers, parents, and citizens who “ …seek principles to critically separate valid from invalid claims for the efficacy and efficiency of education products, personnel, and policies” (p. vii). The book targets classroom teachers who try to improve classroom instruction as well as other educators and policymakers who should be aware of the best means and conditions for students to learn in the school setting. Walberg’s analysis provides readers with factors that enhance and impede student learning according to their proportional learning influences. With Walberg’s conceptualization, educators and policymakers have a useful tool for assessing the efficacy and efficiency of various interventions in relation to available time, money, and expertise. This edited work is a contribution in the Research in Educational Productivity series from the publisher, Information Age Publishing, Inc., and it includes a total of twelve chapters. Although the volume is organized in four parts, this organization was not reflected in the table of contents of the book. Part I can be titled Methodology. It reviews scientific methods for educational research, policy, and practice. Part II concentrates on models and theories that focus on education productivity. This part can be titled Methods and Theories to Educational Productivity. Part III could be titled Application to the Education Profession and Problems. It presents prospective and current applications of scientific methods to education issues. This part introduces the innovative programs that exemplify outstanding uses of rigorous scientific research methods, including the ones which are used in conjunction with qualitative methods. Part IV includes the recommendations of Subotnik and Walberg for policy and practice derived from the individual chapters and face-to-face deliberations conducted at the conference that preceded the publication of the book. It is crucial when facing a policy decision that the research involved is valid and well- designed. The first section of Part I begins with a paper which provides readers with an overview of the key elements of experimental design research as well as the reasons why rigorous research is important to the success of school reform. This chapter outlines the criteria for analyzing the quality of research designs and it rank orders the research design Duyar: Analyzing education productivity: An essay review 11 quality. The readers are offered an invaluable list of important questions to ask when reviewing or conducting research. These questions help assess whether a design is suited to the question or problem at hand. The author urges researchers to put forward their most rigorous efforts at solving the pressing academic problems, although the highest quality designs are often the most difficult to implement. This paper also recognizes the threats to internal and external validity and suggests the use of multiple designs. When searching for validity of results, it is important to keep in mind that the more different designs that are used, the more valid the evidence of the studies becomes. “…the best evidence may be found in the multiplicity of results. No matter how large and well designed, a single experiment may not be definitive. Consistent results from many studies provide the best evidence.” (Paik, 2006, p. 26). Another design discussed in Part I is the “research-based” design. Layng, Stikeleather, and Twyman, the authors of this section, make the distinction between research-based design, which has been designed based on evidence, and research “filtered interventions,” which are still subject to scientific evaluation. The research-based design can be verified by using formative or summative evaluations. Formative evaluation provides an experimentally controlled research base to increase program effectiveness, while summative evaluation provides a statistically controlled research base to increase program effectiveness. These evaluations are also important because they place emphasis on the individual. In fact, this rigorous design uses the same philosophy used in the aircraft building, where manufacturers do not test one aircraft against another; instead, they test each individual plane until it is in perfect condition to fly. This form of formative or research guided evaluation in a single- subject design is introduced for the scientific generation and prediction of education productivity. The final facet of Part I depicts how non-experimental qualitative and descriptive research illuminates experimental and quasi-experimental investigations of reading strategies taught to preservice and inservice teachers. Pang and Kamil believe that there can be a successful mixing of experimental and descriptive research. “…the blending of data from research conducted using different methodologies has the potential for enriching the knowledge base” (Pang & Kamil, p. 58). They promote a future for educational research that carefully delineates the assumptions behind published research, uses mixed methods, and reflects greater familiarity with a range of research paradigms. Part II begins with the introduction of models and theories that improve education productivity. This part addresses the most common questions asked by the policymakers: “what works?” and “what are the effective interventions and policies?” The first section synthesizes psychological research to illustrate how experimental and longitudinal studies can be combined in understanding students’ progress and how to best arrange conditions for their success. Halpern challenges readers to consider critical thinking as a key factor contributing to education productivity. In Halpern’s opinion, teaching for the transfer of critical thinking skills instruction to classroom settings and beyond is the only acceptable outcome of educational interventions. The next section in this part is by Walberg, one of the co-editors of the book. It deserves special attention and discussion. This section provides readers with a comprehensive analysis of substantive research in the field and offers a conceptual framework for education Education Review Vol. 9 No. 4 http://edrev.asu.edu 12 productivity. The author discusses the foundations of widely implemented education reform initiatives in relation to findings of educational research. The analysis of the pertinent research provides policymakers and educators with information they need to make policy to improve the preparation of teachers, and base school reform on scientific evidence. This section also openly criticizes many prevalent and new education policies for being ineffective and inefficient. The author views these policies as distracting and costly fads. The Reading Recovery, Success for All, Title I programs, and special education programs are some of those popular programs which are often chosen by reputation rather than by careful review of evidence of their results and costs. Furthermore, as one of the architects of the current standards and accountability movement, Walberg supports and introduces some of current practices, such as external examinations, accountability for results, and school choice. This section also offers readers a widely known conceptual framework for the study of education productivity. As an educational psychologist, Walberg views learning as the outcome of the education system and searches for factors enhancing or impeding learning. To him, though economic, sociological, and political factors affect learning, their influences are indirect and minimal. Learning is fundamentally a psychological process where student aptitude, quality of instruction, and psychological environment are the well-established, consistent, and proximal causes of learning. The conceptual framework includes nine sub- factors of education productivity. First, student aptitude underlying factor includes (a) student ability or prior achievement, (b) motivation, and (c) developmental level (e.g. age). The elements of instruction involve (d) quantity or amount of instructional time and (e) class environment. Finally, the psychological environment consists of (f) class environment, (g) the stimulating qualities of the home environment, (h) peer environment, and (i) exposure to mass media, particularly television, outside of school. This framework for education productivity has inspired many researchers and has resulted in hundreds of studies. Later studies incorporated the “career aspirations” with Walberg’s framework of educational productivity and recorded even better explanatory power for the expanded model (Ma &Wang, 2001; Young, Reynolds, & Walberg, 1996). These studies showed that education performance is an intermediate factor; there is a strong link between educational outcomes and career aspirations; and factors of education productivity in Walberg’s model have indirect, but significant effects on career aspirations. In addition to its significance in education productivity research, Walberg’s framework also has provided policymakers with a practical list of underlying factors enhancing or impeding student learning. Through the utilization of this list of influences, policymakers may be able to assess the efficacy and efficiency of various interventions and reform initiatives in relation to available time, money, and expertise. The last section of this part argues that research must be based on valid ideas and measures founded in recognized academic disciplines, such as economics and psychology. Sternberg guides readers through the development of the theory of intelligence from its beginnings as a concept, to testing instructional materials based on the theory. At each stage, the author demonstrates the rigorous thinking required to introduce change in schools that will promote both increased achievement and the acquisition of important life skills. The description of the theory testing presented in this section provides readers with a model for subjecting other theories to rigorous tests. Duyar: Analyzing education productivity: An essay review 13 Part III of this book focuses on actual and prospective applications of scientific methods to real problems in education. The first section of this part focuses on the role of the National Research Council in addressing discussions surrounding definitions, organization, and use of high quality education research. The author urges the reader to embrace the challenge of scientifically based educational research and to integrate its concepts throughout the education system with a focus on teacher preparation and professional development. The second section of this part complements the others by emphasizing the importance of the teacher in improving student learning. This section provides readers with an example of applying standards of science to the process of assessing professional preparation and certification for K-12 teachers. It presents in detail the scientific process applied by the American Board for Certification of Teacher Excellence in identifying and assessing research-based teaching skills and content knowledge essential to success as a teacher. The third section of this part offers a comprehensive view of the efforts made by school psychologists to document, organize, and implement their evidence-based practices. Kratochwill views the work of school psychologists as central to providing a holistic picture of education productivity. He models how a group of scholar-practitioners can reflect on science behind all aspects of their profession, from educating, training, evaluating, and testing, to clinically practicing the standards for publication. The final section of this part examines the “What Works Clearinghouse” which was designed to meta-analyze a large number of experiments and quasi-experiments and catalogue the knowledge base for educators and policymakers. The paper describes the screening criteria for choosing studies, methods for synthesizing their collective results, and validating the findings. The work of the Clearinghouse offers policymakers and educators needed evidence related to the effects of interventions, especially evidence that allows causal inferences. Part IV of the book chronicles advice for policy and practice developed from the chapter contributions as well as deliberations conducted at the conference which occurred prior to the publication of the book. A major facet discussed in this part is the necessity of raising standards for educational research and the examination of rigorous forms of research. The varieties of thorough educational research forms included quasi-experiments, randomized experiments, formative research, observations, regression analysis, cost effectiveness analysis, consumer research, and research synthesis. All of these forms of research have their place in improving educational practice and policy when used appropriately and correctly. This section also emphasizes the importance of adequate communication between researchers and educators. This is especially true when educators choose what works and effectively implement selected educational policies. Subotnik and Walberg assert that the communications gap among the stakeholders remains, and thus, educators and their organizations have promoted policies and practices based on inadequate and nonindependent research. They view “action research” as a failed attempt of the 1960s to increase knowledge utilization and research collaboration. They further introduce the initiative “Know That, Know How, Can Do” as a new strategy to enhance the creation and utilization of evidenced-based knowledge among stakeholders. The book and its preceding conference exemplify such a collaboration in the area of education productivity. Education Review Vol. 9 No. 4 http://edrev.asu.edu 14 Overall, The Scientific Basis of Educational Productivity effectively ties in the importance of applying rigorous research methods to enhance the productivity of the American school system. It serves to its main goal exceptionally well, which is to win the trust of the American public in all spheres by testifying to the role and function of thorough educational research in reforming education and improving American students’ achievement. It further addresses the long overdue question of what works and offers policy makers tools to assess the effectiveness of educational interventions and reform initiatives. It further offers a basis for knowing what new educational knowledge and practices can be trusted and a means for reaching consensus within the professional community about the evidence-based best practices. The authors and editors of Scientific Basis of Educational Productivity deserve congratulations for the book's originality, timeliness, and insight into advancing American education productivity. References Anderson, D. M. (1999). Stretching the tax dollar: Increasing efficiency in urban and rural schools. In L. O. Picus & J. L. Wattenbarger (Eds.), Where Does the Money Go? (pp. 156-177), Thousand Oaks, CA: Corwin. Ball, G. C. & Goldman, S. (1997). Improving education’s productivity: Reexamining the system to get the schools we need, Phi Delta Kappan, 79, 228-232. Bishop, J. H. & Wößmann, L. (2004). Institutional effects in a simple model of educational production, Education Economics, 12(1), 17-38. Cook, T. D. (2002). Randomized experiments in educational policy research: A critical examination of the reasons the educational evaluation community has offered for not doing them, Educational Evaluation and Policy Analysis, 24(3), 175-199. Cooper, B. S., Sarrel, R., Darvas, P., Alfano, F., Meier, E., Samuaels, J., et al. (1994). Making money matter in education: A micro-financial model for determining school- level allocations, efficiency, and productivity. Journal of Education Finance, 20(3), 66-87. Dills, A. K. (2005). Does cream-skimming curdle the milk? A study of peer effects, Economics of Education Review, 24(1), 19-28. Duyar, I., McNeal, L., & Kara, O. (2006). Productivity. In English, F. (Edit.), Encyclopedia of Educational Leadership and Administration, pp. 807-809. Thousand Oaks, CA, SAGE. Duyar, I. (2004). Educational Management Organizations in the Delivery and Management of Public Education, Association of School Business Officials International, 90th Annual Meeting and Exhibits, Cincinnati-Ohio, October 15-19. Duyar, I. (1996). Market imperfections in public education, Journal of Educational Administration, Ankara, Pegem, 1(6), 53-67. Harker, R. & Tymms, P. (2004). The effects of student composition on school outcomes, Duyar: Analyzing education productivity: An essay review 15 School Effectiveness & School Improvement, 15(2), 177-199. Hedges, L., laine, R., & Greenwald, R. (1994). Does money matter? A meta-analysis of the effects of differential school inputs on student outcomes, Educational Researcher, 23(3), 5-14. Hill, N. E. & Taylor, L. C.(2004). Parental school involvement and children’s academic achievement, Current Directions in Psychological Science, 13(4), 161-164. Hoxby, C. C. (2004). Productivity in education: The quintessential upstream industry, Southern Economic Journal, 71(2): 209-231. Johnson, K.A. (2000). Do small classes influence academic achievement? What the national assessment of educational progress shows. (CDA Report No. 00-07). Washington D.C., Heritage Foundation. Kara, Duyar, Christy, & McNeil. (2006). Principal-agent theory, In Fenwick, E. (Edit.), Encyclopedia of Educational Leadership and Administration, pp. 796-797. Thousand Oaks, CA, SAGE. Ma, X. &Wang, J. (2001), A confirmatory examination of Walberg’s model of educational productivity in student career aspiration, Educational Psychology, 21(4), 443- 453. Monk, D. (1992). Education productivity research: An update and assessment of its role in education finance reform, Educational Evaluation and Policy Analysis, 14(4), 307-332. National Center for Education Statistics, (2005). Digest of Education Statistics and Tables and Figures, Retrieved on July 11, 2006, from: http://nces.ed.gov/programs/digest/d05/tables/dt05_167.asp National Center for Education Statistics. (2006). The Condition of Education 2006, Washington: DC. Odden, A. (1992). Discovering educational productivity: An organizational approach, Educational Evaluation and Policy Analysis, 14(4), 303-305. Paik, S. J. (2006). Evidence-based reform: Experimental and quasi-experimental research considered, The Scientific Basis of Educational Productivity, pp. 9-28, Greenwich, CT: Information Age Publishing, Inc. Pang, E. S. & Kamil, M. L. (2006). Blending experimental and descriptive research: The case of educating reading teachers, The Scientific Basis of Educational Productivity, pp. 45-84, Greenwich, CT: Information Age Publishing, Inc. Rolle, R. A. (2004). Out with the old-in with the new: Thoughts on the future of educational productivity research, Peabody Journal of Education, 79(3), 31-56. Education Review Vol. 9 No. 4 http://edrev.asu.edu 16 Rolle, R. A. (2003). Getting the biggest bang for the educational buck: An empirical analysis of public school corporations as budget-maximizing bureaus. In W. J. Fowler, Jr. (Ed.), Developments in School Finance 2001-02, (pp. 25-52), Washington, DC: National Center for Educational Statistics. Subotnik, R. F. & Walberg, H. J. (2006). The Scientific Basis of Educational Productivity, (Eds.), Greenwich, CT: Information Age Publishing, Inc. Viadero, D. (2005). Report critiques evidence on school improvement models, Education Week, 25(14), Walberg, H. (1981). A psychological theory of educational productivity. In: F. H. Farley & N. Gorden (Eds.), Psychology and Education, pp. 81–108. Berkeley, CA: McCutchan. Worthington, A. C. (2001). An empirical survey of frontier measurement techniques in education. Education Economics, 19, 245-268. Young, D. J., Reynolds, A. J., & Walberg, H. J. (1996). Science achievement and educational productivity: A hierarchical linear model, The Journal of Educational Research, 89(5), 272-278. Zvoch, K. & Stevens, J. J. (2006). Longitudinal effects of school context and practice on middle school mathematics achievement, Journal of Educational Research, 99(6), 347- 356. About the Reviewer Ibrahim Duyar, Ph.D. Department of Educational Leadership University of Arkansas-Little Rock 2801 South University Avenue Little Rock, AR 72204 Email: Duyar@ualr.edu Dr. Ibrahim Duyar received his doctorate in educational leadership from the University of Wisconsin-Madison in 1995. He has served as a school principal, university professor, and consultant in local, national, and international organizations. He currently teaches politics and policy analysis in educational administration, school finance, and school business management at the University of Arkansas-Little Rock. His research interests include educational policy development and implementation, market-based initiatives in public education, economic aspects of education, and systemic school reform. Duyar, I., McNeal, L., & Kara, O. (2006). Productivity, In English, F.. (Edit.), Encyclopedia of Educational Leadership and Administration, pp. 807-809. Thousand Oaks, CA, SAGE. Duyar, I. (2004). Educational Management Organizations in the Delivery and Duyar: Analyzing education productivity: An essay review 17 Management of Public Education, Association of School Business Officials International, 90th Annual Meeting and Exhibits, Cincinnati-Ohio, October 15-19. Duyar, I. (1996). Market imperfections in public education, Journal of Educational Administration, Ankara, Pegem, 1(6), 53-67. Duyar, I. (1989). Internal efficiency in education, Journal of College of Educational Sciences, 22(1), 485-504, Ankara University Publications.