22 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Economic Education Production Functions for the Principles of Macroeconomics and the Principles of Microeconomics: Is There a Difference? Hart Hodges1, Yvonne Durham2, and Steve Henson3 ABSTRACT This paper examines the relationships among student attitudes towards economics, performance, and knowledge retention in introductory economics courses. We find that the economic education production functions differ across the introductory microeconomics and macroeconomics courses. Attitude affects performance in microeconomics, but not in macroeconomics, while performance does not impact ending attitude significantly in either course. Classroom experiments and gender affect performance directly in macroeconomics, but only indirectly through attitude in microeconomics. The existence of differing production functions provokes important questions about how each of the introductory courses should be designed and taught, suggesting that it is not appropriate to treat all introductory courses in the same way. Additionally, we find that the retention of knowledge depends primarily on past performance and native ability, with classroom experiments slowing the depreciation rate. Key words: Economics Education; Attitude; Performance; Retention; Production Functions JEL classification: A20, A22, I21 Introduction A substantial literature in economic education explores the factors that influence academic success in undergraduate introductory economics. These include such variables as student attitude, aptitude, attendance, effort, learning styles, personality types, gender, and class size. The specific relationships between many of these factors and various measures of performance have not been conclusively established. This may be due, in part, to the fact that the courses from which the data have been drawn have not been uniform. The courses are of a variety of formats, typically Principles of Macroeconomics, Principles of Microeconomics, or a combined Principles of Economics. If the factors influencing student success differ across these courses, the choice of course examined likely impacts the nature of the relationships observed. Using a data set from an earlier study, we provide some insight into the relationships among, and determinants of, three outcome measures: attitude, performance, and retention. While including several of the factors indicated above as possible determinants, we explore the effects of classroom experiments on these three measures, and we do so separately in the introductory macroeconomics and microeconomics courses. We find that educational production functions do indeed differ across these courses, which may help explain prior conflicting results. 1 Associate Professor, Department of Economics, Western Washington University, 516 High St, Bellingham, WA 98225 2 Professor, Department of Economics, Western Washington University, 516 High St, Bellingham, WA 98225 3 Professor, Department of Economics, Western Washington University, 516 High St, Bellingham, WA 98225 23 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Untangling the possible relationship between attitude toward economics and performance is challenging, and the specific nature of that relationship has not been conclusively determined. Does a better attitude toward a particular subject lead to better academic performance, or does better performance lead to a better attitude? Does the causality run in both directions? Or are both attitude and performance jointly determined by other factors? And how might the retention of knowledge be related to these factors and how they interact? This study differs from previous research in two important ways. First, it is comprehensive, examining all three outcome measures. Second, it explores the possibility of differing education production functions in the two principles courses, which to our knowledge has not yet been explicitly considered. The possibility of differing production functions provokes important questions about how each of the introductory courses should be designed and taught. We attempt to distinguish whether the factors that influence learning are the same and if they affect learning in the same way and to the same extent across the two courses. The results indicate that they do not. In particular, ending attitude affects performance in microeconomics but not in macroeconomics, and the mechanisms through which the use of classroom experiments affects performance appear to differ across the courses. In macro, for example, both gender and participation in classroom experiments directly affect performance; but in micro, these variables affect performance only indirectly, through their impacts on attitude. In addition, we find that retention of economic knowledge by business majors is determined by past performance, student aptitude, and time; and that participation in classroom experiments reduces the rate at which the stock of economic knowledge depreciates. We find no evidence that attitude toward economics affects retention of economic concepts directly; however, there may be indirect effects from a linkage between attitude and performance. In the following section, we review the literature on educational production functions and the factors influencing academic success in introductory economics. Next we describe the model and data used in the present analysis, and we then discuss the results and their implications. Background Microeconomics versus Macroeconomics Principles textbooks generally distinguish between microeconomics and macroeconomics on the basis of the unit of exploration. From the student viewpoint, there appear to be two additional distinctions that may be important determinants of performance: differences in perceived course difficulty and subject matter appeal. Student comments on various internet forums, and our own conversations with students, provide some insight into how some of our students may distinguish between the two courses on these other levels. Students view micro as more difficult, involving more mathematics, graphs, and coverage of less familiar topics, and macro as more data driven and interesting because it explores more practical, applicable, and familiar concepts that appear in the news. The notion that students may find micro to be harder, less applicable, and less interesting is also consistent with previous research. For example, Benedict and Hoag (2002) find that students taking microeconomics are more likely to be apprehensive than those taking macro, perhaps because micro has the reputation of being more theoretical and less policy-oriented. Hamermesh (2002) indicates that microeconomics can seem quite dry to students and only comes alive when instructors provide engaging examples that illustrate the key principles. Research also provides some insight into differences in student performance across the two principles courses. Grimes and Nelson (1998) find that mastering microeconomic principles 24 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 appears to require more technical expertise than mastering macroeconomics. Surprisingly, Cohn et al. (2001) observe that in a Principles of Macroeconomics course there may be circumstances in which the use of graphs may be counterproductive, and suggest that their use may persist in large part because instructors prefer them. Colander (2000) indicates that “…a large portion of our audience does not know the language of models, mathematics.” If micro is more theoretical, abstract, and graph-oriented, and if students struggle with graphs and models, then student perceptions that micro is harder than macro might be valid. In addition, one could argue that even some of the most basic concepts that form the foundation of the rational decision-making process studied extensively in microeconomics are initially counterintuitive to students. In their text, Frank, Bernanke, Antonovics, and Heffetz (2016) discuss some “important decision pitfalls,” incuding ignoring foregone opportunities, failing to ignore sunk costs, and not thinking at the margin. Behavior that violates the basic tenets of microeconomic theory is common enough that the field of behavioral economics has arisen in order to explore these anomalies. So basic concepts that we often assume are intuitive or “common sense” to our students in micro may not be, requiring additional effort from both students and instructors. Conversely, many macroeconomic ideas may tend to be more intuitive and familiar to our students. Topics such as GDP, inflation, unemployment, and the Federal Reserve are often in the news. It would not be surprising to find that students struggle more, both from an interest and intellectual perspective, to grasp some of the more counterintuitive, technical, and less familiar ideas used in the study of microeconomics. Given these disparities, it is plausible that learning occurs differently in these two courses, resulting in distinct educational production functions. Attitude and Success in Economics In a seminal article, Bach and Saunders (1965) suggest that the understanding of economic ideas may depend on a student’s interest in economics. A series of studies followed, attempting to discover whether such a relationship actually exists, and if so, its nature. Given the possible complexity of this interaction between attitude and performance, it is not surprising that the assorted specifications and choices of courses examined have resulted in differing conclusions. Some studies find that attitude affects achievement (Karstensson and Vedder, 1974; Wetzel, Potter, and O’Toole, 1982; Chizmar and McCarney, 1984; Brock, 2011); but others (Manahan, 1983; Hodgin, 1984; Charkins, O’Toole, and Wetzel, 1985) find that it does not. There are also conflicting findings for the effect of achievement on attitude. The Effects of Learning Styles and Personality Types Several studies have examined the impacts of student learning styles, instructor teaching styles, and student and teacher personality types. Many studies find improved performance and attitudes when learning and teaching styles or teacher/student personality types are better aligned (Wetzel, Potter, and O’Toole, 1982; Charkins, O’Toole, and Wetzel, 1985; Borg and Shapiro, 1996; Borg and Stranahan, 2002; Terregrossa, Englander, and Wang, 2009). Ziegert (2000) finds that personality differences may be partially responsible for the often observed “gender gap” discussed below. Gender Effects The role of gender in economic learning has not been resolved conclusively. Much of the existing empirical work finds that males do better (Dancer, 2003; Jensen and Owen, 2001; Robb 25 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 and Robb, 1999; Feiner and Roberts, 1995; Ferber, 1995; Anderson, Benjamin, and Fuss,1994). However, the relationship between gender and measures of economic learning appears to be complex. MacDowell, Senn and Soper (1977) find that the gender effect found in college-level economics courses is not present in younger students. Lumsden and Scott (1987) offer evidence that gender effects depend on the testing format. Multiple-choice exams are often used in the principles courses, and females tend to perform better on written tests. Benedict and Hoag (2002) also find females to be more apprehensive about economics courses than their male counterparts. The Effects of Classroom Experiments While also not conclusive, there is some evidence that classroom experiments can improve performance (Gremmen and Potters, 1997; Frank, 1997; Dickie, 2006; Durham, McKinnon, and Schulman, 2007). Cardell et al. (1996) and Emerson and Taylor (2004) get mixed results that depend on the university/course studied and the measure of performance used. Yandell (1999) finds that increased use of classroom experiments does not significantly affect performance. Durham, McKinnon, and Schulman (2007) find that classroom experiments in the principles courses significantly affect performance on the specific material covered by the experiments, attitude toward economics, and retention, with a larger overall impact on experimental content performance in macroeconomics. Retention as a Measure of Success In addition to performance measures such as course/exam grades and scores on the Test of Understanding of College Economics (TUCE), another important measure of academic success is student knowledge retention over time. As with the other performance measures, many possible determinants have been explored and the results are far from conclusive. Walstad and Allgood (1999) and Walstad (2001) find evidence of fairly low retention rates for economic knowledge, while Saunders and Bach (1970) and Crowley and Wilton (1980) are more optimistic. Research has explored the effects on retention of a wide range of variables including attitude, SAT scores, GPA, time, original course grade, age, gender, major, and course format. Conclusions have varied widely (Saunders and Bach, 1970; Craig and O’Neill, 1976; Kohen and Kipps, 1979; Kane and Spizman, 1999). Since a compelling argument can be made that retention is one of the most important performance measures, determining what inputs are significant in the retention production function is an important task. The Current Study This study focuses on three things. First, we consider the relationship between student attitudes towards economics and performance, giving special attention to the effects of classroom experiments while controlling for student learning styles, demographic characteristics, attendance, and aptitude. Second, we allow the production functions to differ between the introductory microeconomics and macroeconomics courses, yielding insights that have not been available from previous studies that did not consider this distinction. Finally, we explore the determinants of retention of introductory economic concepts. Model Specification We hypothesize a joint production process in which a student’s performance in, and attitude towards, economics at the end of the semester are simultaneously determined throughout the course by four broad sets of variables: (1) the student’s stock of human capital at the beginning 26 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 of the course, (2) other individual-specific characteristics and experience that affect the student’s attitude toward the subject and/or ability to convert that human capital into performance, (3) a flow of inputs (effort) provided by the student, and (4) inputs provided by the institution. Retention depends on these factors and on the student’s performance in the course. Specifically, let A be a student’s attitude toward economics at the end of the course, P be performance as measured by course grade, and R be a measure of retention of the course material at some future date. We estimate a model of the form: (1) A i = γ 1 P i + X i β 1 + Z 1i α 1 + u i (2) P i = γ 2 A i + X i β 2 + Z 2i α 2 + ε i (3) R i = γ 3 P i + W i β 3 + v i . Here X is a set of exogenous variables believed to directly affect both ending attitude and performance concurrently, such as demographic characteristics, effort (proxied by course attendance), and institutional inputs such as class size and whether experiments were used. The matrix Z 1 contains variables that directly affect ending attitude toward economics but not performance. We hypothesize this to be the student’s attitude at the beginning of the course. A student’s initial attitude toward economics is the starting point from which attitude evolves as the course progresses, and is thus likely to be correlated with ending attitude. However, unlike in more familiar subjects from high school such as math and history, students often begin an economics course, particularly micro, with little or no prior knowledge about the subject matter. Their relatively uninformed initial impressions of economics are likely to be only weakly related to unobserved variables that affect performance in the course. We thus use beginning attitude as an instrument for ending attitude in estimating the performance equation. The other set of instruments, Z 2 , contains human-capital variables that are likely to affect the course grade but are unlikely to have a direct influence on ending attitude. As with attitude, ability is not observed directly. We proxy ability by using standardized test scores (ACT or SAT) and cumulative GPA prior to the course. One question that might arise about the appropriateness of using general academic ability as an instrument for performance is that more-capable students might have overall better attitudes toward learning, so that past performance variables might be correlated with the error term in equation (1). This concern is mitigated by the fact that our attitude variables measure student attitude toward economics specifically rather than toward learning in general. Additionally, several questions in the attitude survey ask the student to compare their view of economics with the other subjects they are studying. There is no reason to suspect that more- capable students would view economics in a better light than their other subjects. In the retention equation, W contains many of the same variables as X, as well as a measure of the time elapsed between the economics course and the retention test. The presumption that both ending attitude and course grade are interdependent and jointly determined means that ordinary least squares estimates of equations (1) and (2) may be biased and inconsistent, necessitating estimation by two-stage least squares. We test this hypothesis below. No such issue arises with equation (3), for which all right-hand-side variables are predetermined. 27 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Data The data used in this study come from a controlled experiment that was conducted at the University of Arkansas from the fall semester of 2000 through the fall of 2002. While the experiment was conducted for the purpose of ascertaining the effects of classroom experiments on learning, information was collected on the three performance variables in which we are interested and many of the factors thought to affect them, for both introductory courses. The original study used a treatment/control group design, with the treatment (classroom experiments used) and control (no experiments used) groups separated across semesters to allow for control of confounding factors such as the instructor and the time, duration, and size of the class. The study used two sections each of introductory micro and macro per semester for four semesters. The four fall semester sections each year were control sections; the four sections in the spring semester used classroom experiments. All sections of both courses in the first year were 120-seat sections, and all in the second year were 60-seat. Class times were held constant across treatment and control sections, with all macro courses taught on Tuesdays and Thursdays and all micro courses taught on a Monday-Wednesday-Friday schedule, all between 8:00 and 11:00 am. The control sections were taught using a traditional lecture/class discussion format, while the treatment sections supplemented lectures with classroom experiments designed to illustrate a key basic economic concept.4 The pacing of the material was closely matched between control and treatment groups so that each received approximately the same amount of contact time for a particular topic area and no additional homework was assigned over the experiments. In the control group, additional discussion, examples, and problems were used instead of experiments. The macro sections were taught by one instructor and the micro sections by two other instructors, all of whom had limited experience using classroom experiments. The text used in the micro courses was similar in the technical/mathematical level of topics to that of the macro text.5 Students in the study completed a survey at both the beginning and end of each course to assess their attitudes toward economics. The survey asked students to respond, on a five-point scale from “very poor” (1) to “very good (5),” to a set of eight questions about their interest in and attitude toward economics currently, their interest in learning more and taking more courses, and how their interest in economics compared to other subjects. The attitude variable used in the analysis is the average of these eight rankings.6 Subsequently, in an upper-division finance course that had both principles courses as prerequisites, students from both the treatment and control sections were given a pretest that incorporated a basic question from each of the 13 topic areas covered by the experiments. Student performance on this pretest allows us to assess retention of material from the earlier classes and to 4 Micro sections used eight experiments and macro used five. The topic areas covered by the micro experiments were resource allocation, comparative advantage, demand and supply, diminishing marginal utility, production and costs, monopoly, cartels, and public goods; and macro covered equity versus efficiency, savings and consumption, money creation, CPI bias, and the federal funds market. Descriptions of these experiments are available on request. Given the nature of the material in the two courses, it is reasonable to expect that a different number of experiments might be appropriate for the different courses. Experiments in macro can often be more complex and lengthy, and several of the micro experiments employed were fairly simple and short. While the number of experiments implemented differed, the total class time spent on them was likely similar. 5 While each instructor was asked to introduce the experiments, they chose the other material to cover. All three instructors used analytical/technical tools as the backbone of their courses, but did so with many examples and policy discussions to motivate the models. 6 For feasibility reasons, the attitude survey was administered during the last week of classes, prior to the final exam and determination of course grade. A copy of the attitude survey is available on request. 28 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 examine how retention might be affected by student characteristics, the use of classroom experiments, and other variables of interest. The original dataset contained 753 students in eight sections of macro, 727 in eight sections of micro, and 1,012 students in 18 sections of finance. After eliminating cases with missing data, we were left with usable samples of 466 students in macro, 309 in micro, and 268 in the retention file.7 Sample means and standard deviations are presented in Table 1. The “Retention” column refers to students in the upper-division finance course used for the retention analysis. Several aspects of the data are worth noting. First, consistent with the previous discussion of students’ perceived differences between microeconomics and macroeconomics, students generally found macro to be a more enjoyable experience than micro. Not only are the beginning and ending attitudes both higher for macro than for micro, but the change in attitude from beginning to end of the course is significantly positive in macro (t = 9.63) and negative in micro (t = –5.06). This is also consistent with the overall higher course grades in macro than in micro.8 At the University of Arkansas, students are not required to take the introductory courses in any particular order. However, the sequencing of course numbers places macro before micro, so many students take them in that order.9 It may be that some students have a fairly positive experience in macro that sets them up for disappointment in what they perceive to be more difficult material in micro.10 During the time period in which the data were collected, approximately 97% of all students at the University of Arkansas submitted ACT scores in support of their admission application, and the remaining 3% submitted scores on the SAT. In order to avoid losing the SAT portion of the sample, we standardize each of these scores by subtracting its mean and dividing by its standard deviation. We do this separately for the quantitative and verbal portions of the tests, and use these standardized variables zactsat_math and zactsat_verbal as explanatory variables in the performance equation. The variables dlsv, dlsa, dlsr, and dlsk reflect student responses to the VARK learning- styles questionnaire (Fleming and Bonwell (1998)). VARK stands for visual, aural, read/write, and kinesthetic. The variables sophomore, junior, and senior are defined as of the quarter during which the macro or micro class is taken. The reference category is freshman, which make up 15.0% of the students in macro, 14.2% in micro, and 8.2% in finance. 7 In macro, there were final course grades for 700 students, suggesting that 53 (7.7%) had dropped the course during the semester. In micro we had final grades for 646, suggesting 81 drops (11.6%). The remainder of the missing observations are due to missing data on other variables, most commonly ending attitude, followed by beginning attitude and learning style. The retention file contains only students who were in either a treatment or control section of the macro or micro courses that were part of the study. For additional details on the experimental design, data collection procedures, and analysis of the effects of classroom experiments, see Durham, McKinnon, and Schulman (2007). 8 For the null hypothesis that mean grades are the same between micro and macro, t = 1.95 (p = 0.052 with n = 775). For the subsample of 76 students who took both courses during the study period, t = 0.6176. 9 At the University of Arkansas, the Principles of Microeconomics and Principles of Macroeconomics courses can be taken to fulfill the social science portion of the University Core Requirements. So while these courses are not required for most non-business majors, they can be used to fulfill a graduation requirement. 10 The sequencing of courses probably explains why students in microeconomics are on average slightly older and more likely to be business majors, for whom the course is required. 29 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Table 1 Summary Statistics: Sample Means (Standard Deviations in Parentheses) Variable Description Macro Micro Retention gp Grade in course (0-4) 2.616 (0.939) 2.479 (0.985) 2.695 (0.802) endatt Ending attitude toward economics: 0 (very poor) – 4 (very favorable) 3.946 (0.726) 3.222 (0.849) begatt Beginning attitude toward economics: 0 (very poor) – 4 (very favorable) 3.640 (0.685) 3.453 (0.689) busmajor 1 if business major, 0 otherwise 0.687 (0.464) 0.835 (0.372) cumgpa Cumulative grade-point average at beginning of course (0-4) 2.919 (0.632) 2.839 (0.619) 3.015 (0.517) zactsat_math Standardized value of ACT or SAT quantitative score 0.000 (0.999) 0.000 (0.999) 0.000 (0.998) zactsat_verbal Standardized value of ACT or SAT verbal score 0.000 (0.999) 0.000 (0.999) 0.000 (0.998) male 1 if male, 0 otherwise 0.549 (0.498) 0.602 (0.490) 0.571 (0.496) minority 1 if African-American, Hispanic, or Native American; 0 otherwise 0.071 (0.257) 0.081 (0.273) 0.063 (0.244) age Age in years 19.356 (1.600) 20.298 (2.255) dlsv 1 if learning style is “visual,” 0 otherwise 0.030 (0.171) 0.032 (0.177) 0.019 (0.136) dlsa 1 if learning style is “aural,” 0 otherwise 0.088 (0.284) 0.087 (0.283) 0.101 (0.312) dlsr 1 if learning style is “read/write,” 0 otherwise 0.054 (0.226) 0.036 (0.194) 0.063 (0.244) dlsk 1 if learning style is “kinesthetic,” 0 otherwise 0.212 (0.409) 0.201 (0.401) 0.198 (0.399) sophomore 1 if credits earned is at least 30 but less than 60, 0 otherwise 0.618 (0.486) 0.528 (0.500) 0.631 (0.484) junior 1 if credits earned is at least 60 but less than 90, 0 otherwise 0.185 (0.388) 0.288 (0.454) 0.276 (0.448) senior 1 if credits earned is at least 90 , 0 otherwise 0.047 (0.212) 0.042 (0.201) 0.011 (0.105) attend Proportion of classes attended 0.804 (0.170) 0.713 (0.201) largeclass 1 if 120-seat class, 0 if 60-seat 0.719 (0.450) 0.647 (0.479) treatment 1 if classroom experiments used, 0 if not 0.491 (0.500) 0.531 (0.500) instruct Binary indicator assigned to one of two micro instructors 0.440 (0.497) nlargeclasses Number of large classes in introductory economics 1.813 (0.409) ntreatments Number of treatment sections in introductory economics 0.627 (0.515) recent 1 / (number of months since most recent introductory economics class) 0.203 (0.162) score Proportion of correct answers on retention test 0.505 (0.145) Observations 466 309 268 30 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Attend is the proportion of class sessions that a student attended on randomly-selected days throughout the semester. The number of days sampled varied from class to class. This raises some doubt about the reliability of this variable and its comparability across different students. Nevertheless, there are at least two arguments for its inclusion in the analysis. First, it is the only measure of student effort that is available to us, and its omission could bias the coefficients on other variables in the model to the extent that those variables are correlated with attendance. Second, if measurement error in this variable is truly random, then we would expect its estimated coefficient to be biased toward zero. This is worth noting as we consider the results below.11 Institutional inputs are captured by the variables largeclass, treatment, and instruct. The class size variable is equal to 1 if the class has 120 students rather than 60; treatment controls for whether the class included experiments; and instruct is a dummy variable that indicates which of two instructors taught the micro course.12 The bottom four rows in Table 1 are unique to the retention data. We define the variable recent to be the inverse of the number of months between the end of the most-recent introductory economics class and the beginning of the finance class in which retention is measured. It ranges from 1, for students who took their final principles course in the fall and then finance in the spring semester of the same academic year, to 0.05, for students who took the finance course 20 months after the principles course. As for the other variables in the “Retention” column, cumgpa, zactsat_math, zactsat_verbal, male, and minority were recorded at the beginning of the finance course; and gp and the learning-style variables in that column are an average of the values from the student’s micro and macro courses.13 The variable ntreatments is the sum of the number of macro and micro classes in which the student was exposed to experiments. This variable is equal to zero for 104 students. For 160 students it is equal to one, indicating experiments were used in either the micro or macro section that they took. It is equal to two for four students who were in sections of both micro and macro in which experiments were used. Results Following the organization of the Background section above, we first present results on the relationship between attitude and performance and then discuss the effects of other variables such 11 Since we do not have transcript information for the students in the study, we cannot determine if a student had previously taken the course and was repeating it. We only have this information for those students who had previously taken a section of the course that was included in the study. In macro, there were no students who had previously completed one of our macro sections, and only one who had previously enrolled but did not finish. In micro, six students had previous enrollments in one of our micro sections and three of these had completed the course. We examined some specifications that included dummy variables for previous enrollments or completions. None of these variables was statistically significant, and their inclusion had only minor effects, if any, on a few of the other coefficients at the second or third decimal place. Results are available upon request. 12 Because the data used in this study were gathered for a different objective, the experimental design controls for instructor across treatment and control sections but does not do so across the two courses. While this may seem problematic at first glance, a compelling argument can be made that it is actually appropriate for the current study. Educational research supports an argument that it is not the specific person serving as the instructor that needs to be controlled, but rather, the level of expertise with which the particular course is taught. (See Shulman (1986) and Ball, Thames, and Phelps (2008)). The three faculty involved in the study each had a great deal of experience and expertise teaching the specific principles course for which they were the instructor. The same claim cannot be made with regard to their experience and expertise in teaching the other principles course. 13 This is necessary because separate scores for the micro and macro questions on the retention test were not available in the data. 31 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 as classroom experiments. We do not include a discussion of learning styles, as the learning style variables were not significant in any of the equations14 Attitude and Performance Ordinary least-squares estimates of the attitude equation are presented in columns 1 and 3 of Table 2. Not surprisingly, beginning attitude toward economics is an important determinant of ending attitude in both macro and microeconomics. Business majors have better attitudes toward economics at the end of their macro course than do their peers, but there is no significant difference in micro. On the other hand, males have better attitudes than females at the end of their micro course, but there is no gender difference in macro. Participating in experiments appears to have a positive attitudinal impact for students in microeconomics but not in macro. Ethnicity, age, learning style, and class size have no statistically significant effect on final attitude. The possibility that attitude and performance are interdependent and jointly determined raises the possibility that performance (measured by gp) may be correlated with unobserved variables that affect ending attitude, resulting in biased and inconsistent OLS estimates. To test this possibility, we estimate equation (1) by two-stage least squares using cumgpa, zactsat_math, and zactsat_verbal as instruments for gp. Results are given in columns 2 and 4. The row labeled “First-stage F” tests the joint significance of the instruments cumgpa and the two zactsat variables in the first-stage regression. Based on this statistic, we reject the hypothesis of weak instruments in both the macro and micro regressions.15 The insignificant Basmann (1960) 2 statistic indicates that we cannot reject the joint hypothesis that the model is correctly specified and that the instruments are exogenous. The Hausman-Wu test (Hausman (1978), Wu (1973)) easily rejects the hypothesis that gp is exogenous in both the micro and macro equations. The IV results differ from the OLS estimates in two important respects. First, the apparent importance of gp as estimated by OLS disappears when it is appropriately treated as endogenous. This conflicts with the notion that students’ attitudes toward a subject may be affected by how well they perform in it. As discussed earlier, the previous findings on the nature of this relationship are inconclusive. Second, attendance continues to be statistically insignificant in micro, but becomes a strong predictor of ending attitude in macro. In addition, treatment is a predictor of ending attitude in micro but not in macro, but the effect is significant at only the 0.10 critical level. Table 3 presents the results for the performance equation. In microeconomics, the OLS and IV estimates are virtually identical, so the Hausman-Wu test fails to reject the hypothesis that endatt can be treated as exogenous. In macro, however, the post-estimation tests clearly favor the IV results. In what follows, therefore, we focus on the IV results.16 14 However, given their theoretical relevance, we retain these variables in the models to avoid any potential bias that might result from their joint exclusion. Though they are individually insignificant, the learning style variables together account for 22-24% of the explained variance of endatt and gp. 15 For a discussion of this rule of thumb, see Staiger and Stock (1997) and Stock, Wright, and Yogo (2002). 16 The Basmann chi-square test was not performed because the performance equation is exactly identified, so there are no overidentifying restrictions to test. 32 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Table 2 Dependent Variable: Ending Attitude (1) (2) (3) (4) Macro Macro Micro Micro VARIABLES OLS IV OLS IV gp 0.154*** -0.031 0.218*** 0.024 (0.036) (0.052) (0.047) (0.069) begatt 0.561*** 0.565*** 0.500*** 0.543*** (0.042) (0.043) (0.060) (0.061) busmajor 0.186*** 0.131** 0.174 0.172 (0.065) (0.066) (0.113) (0.113) male 0.053 0.050 0.354*** 0.351*** (0.057) (0.058) (0.086) (0.085) minority -0.049 -0.088 -0.177 -0.195 (0.111) (0.112) (0.151) (0.150) age 0.032 0.026 -0.015 -0.024 (0.022) (0.022) (0.019) (0.019) dlsv -0.037 -0.016 -0.256 -0.283 (0.167) (0.169) (0.231) (0.231) dlsa 0.122 0.137 0.127 0.112 (0.101) (0.102) (0.146) (0.146) dlsr -0.151 -0.142 -0.079 -0.109 (0.126) (0.128) (0.212) (0.212) dlsk -0.000 0.020 -0.007 -0.039 (0.073) (0.073) (0.105) (0.105) sophomore -0.090 -0.046 -0.023 0.025 (0.086) (0.087) (0.123) (0.123) junior -0.075 0.012 0.020 0.105 (0.104) (0.106) (0.135) (0.137) senior -0.280* -0.216 -0.043 0.058 (0.163) (0.165) (0.235) (0.236) attend 0.134 0.630*** -0.011 0.450 (0.203) (0.228) (0.248) (0.274) largeclass 0.044 0.047 -0.124 -0.061 (0.074) (0.074) (0.095) (0.096) treatment 0.059 0.092 0.152* 0.145* (0.061) (0.062) (0.084) (0.084) instruct 0.079 0.057 (0.086) (0.086) Constant 0.633 0.806* 0.905* 1.022** (0.464) (0.470) (0.509) (0.509) Observations 466 466 309 309 R-squared 0.343 0.304 0.359 0.322 First-stage F 142.1*** 87.29*** Basmann chi-square 1.814 Hausman-Wu F 26.92*** 15.99*** Note: *** {**} [*] represent statistical significance at the 1% {5%} [10%] level. Instruments for GP: cumgpa zactsat_math zactsat_verbal 33 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Table 3 Dependent Variable: Course Grade (1) (2) (3) (4) Macro Macro Micro Micro VARIABLES OLS IV OLS IV endatt 0.221*** 0.069 0.302*** 0.301*** (0.036) (0.069) (0.042) (0.088) cumgpa 0.930*** 0.934*** 1.037*** 1.037*** (0.053) (0.053) (0.072) (0.070) zactsat_math 0.065* 0.053 0.010 0.010 (0.036) (0.037) (0.047) (0.046) zactsat_verbal 0.079** 0.077** 0.037 0.037 (0.035) (0.035) (0.044) (0.042) male 0.158*** 0.170*** 0.048 0.048 (0.055) (0.055) (0.075) (0.078) minority 0.050 0.025 0.001 0.001 (0.101) (0.101) (0.127) (0.126) age 0.073*** 0.078*** 0.030* 0.030* (0.020) (0.020) (0.017) (0.016) dlsv 0.067 0.095 -0.102 -0.102 (0.151) (0.151) (0.195) (0.189) dlsa 0.021 0.024 -0.063 -0.063 (0.092) (0.092) (0.124) (0.121) dlsr -0.046 -0.064 -0.107 -0.107 (0.115) (0.115) (0.181) (0.176) dlsk 0.103 0.107 -0.117 -0.117 (0.066) (0.066) (0.088) (0.085) sophomore 0.101 0.084 0.119 0.119 (0.077) (0.078) (0.103) (0.100) junior 0.074 0.053 0.196* 0.196* (0.095) (0.095) (0.114) (0.111) senior -0.078 -0.169 0.189 0.189 (0.148) (0.152) (0.197) (0.191) attend 1.327*** 1.403*** 0.841*** 0.842*** (0.174) (0.177) (0.204) (0.200) largeclass -0.140** -0.123* 0.294*** 0.294*** (0.067) (0.068) (0.079) (0.077) treatment 0.209*** 0.201*** -0.095 -0.095 (0.054) (0.054) (0.071) (0.070) instruct -0.137* -0.137* (0.072) (0.070) Constant -3.649*** -3.196*** -2.847*** -2.845*** (0.450) (0.482) (0.451) (0.500) Observations 466 466 309 309 R-squared 0.678 0.664 0.662 0.662 First-stage F 166.3*** 79.87*** Hausman-Wu F 6.834*** 0.0001 Note: *** {**} [*] represent statistical significance at the 1% {5%} [10%] level. Instruments for ENDATT: begatt 34 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 As in Table 2, there are some notable, and in some cases surprising, differences between the two courses. Ending attitude is a significant predictor of performance in microeconomics, but not in macro. In micro, students in the 120-seat classes have higher final grades than those in the 60-seat sections, but in macro the opposite is true. We suspect this is related to the teaching experiences of the instructors in these courses. One could argue that both class sizes are “large,” so this is not really a comparison between small and large classes, but rather a comparison between “large” and even “larger” classes. The macro instructor had a lot of experience teaching “larger” classes, while both micro professors had more experience teaching “large” sections. Males and students who participated in classroom experiments do better than their counterparts in macro but not in micro. Interestingly, a student’s score on the verbal portion of the ACT or SAT is a predictor of performance in macro but not in micro. Performance on the quantitative part of the college admission test has no statistically significant impact in either course once endogeneity of attitude is taken into account. This suggests that whatever perceptions students may have about micro being more difficult because they perceive it to be more mathematical, their performance in micro is unrelated to their actual quantitative ability as measured by the college admission tests. And performance in macro appears to depend more on students’ verbal skills than on their mathematical ability. Age, attendance, and cumulative GPA are significant determinants of performance in both classes. The impact of age is about twice as large in macroeconomics as in micro. The strong positive effect of attendance is notable given the possibility, noted above, of measurement error in this variable and the consequent attenuation of the coefficient estimate toward zero. Ethnic background and learning style do not seem to matter. Several conclusions emerge from these results. First and foremost, the determinants of, and the relationship between, attitude and performance differ between the two principles courses. Students master the concepts of and form attitudes toward economics differently in macroeconomics than they do in micro. Second, a student’s attitude towards economics at the end of the course does not appear to be affected by that student’s actual grade; and the final grade appears to be affected by final attitude in micro, but not in macro. The importance of attitude in determining performance in microeconomics may be due in part to the technical requirements of the course, with a better attitude providing the incentive to invest more time and energy in learning the material. Third, the most important determinant of a student’s attitude toward economics at the end of the course is the student’s beginning attitude. Fourth, the most significant predictors of performance are cumulative GPA, age, and attendance. And students’ performance on the verbal portion of the ACT or SAT is a predictor of their performance in macro, but neither the verbal or quantitative scores appear to predict performance in micro. These conclusions taken together suggest that both attitude and performance are largely determined by student characteristics that are beyond the control of university administrators or instructors. Indeed, the effects of university inputs are mixed. Class size has no statistically significant effect on attitude. Its impact on performance is measurable only in microeconomics and again, because the class sizes discussed here could both be construed as “large,” the results from the micro course may not be as paradoxical as they might appear. The instructor effect on grades in micro is small and could easily be due to a minor difference in grading standards. Gender The results indicate that gender affects performance in macro directly, but in micro only indirectly through its effect on attitude. Because their attitude is more favorable in micro, males 35 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 may be more willing and interested in investing the necessary time to master the more abstract concepts and required technical skills. Experiments We find that experiments enhance students’ attitudes in microeconomics, and contribute directly to performance in macro. In micro, the effect of experiments on performance is indirect, via their positive influence on attitude, which in turn affects performance.17 While the reasons for this are unclear, one story that is consistent with these results relies on the distinctions discussed earlier. Microeconomic concepts tend to be less familiar and initially intuitive, but arguably more technical than macroeconomic concepts. Thus while experiments help students directly relate to and understand the macro concepts, they might not help with the technical and analytical skills needed in micro. Rather, micro experiments may make the concepts more familiar, the students more comfortable, the problems more interesting, and/or the models more applicable. This likely makes learning economics more attractive and reduces apprehension, causing students to be more confident and willing to do the work necessary to learn the material. In short, the experiments may help provide students with the motivation to make the necessary investment in time and/or effort to learn the material in the micro course. Retention Table 4 presents results from the retention equation. In column 1, the strongest predictors of retention are the student’s overall academic ability as measured by the verbal portion of the college admission test (at the 0.01 critical level) and performance in introductory economics (at the 0.10 level). The only other variable that is statistically significant (at the 0.10 level) is the number of treatment sections.18 The time elapsed since the last introductory class also appears to have no effect on retention. When the number of treatment sections is interacted with recent, however, a more interesting picture emerges. In column 2, classroom experiments again have a direct positive impact on retention. We also see that the statistically insignificant effect of recent in column 1 masks different rates of depreciation between the treatment and control groups. We estimate a depreciation rate of 4.2 percentage points per year in the retention score for students who had no exposure to experiments in their introductory courses. For students with at least one course using 17 The effect is also numerically smaller. Multiplying the relevant coefficients, for micro we get 𝜕𝑔𝑝 𝜕𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 = 𝜕𝑔𝑝 𝜕𝑒𝑛𝑑𝑎𝑡𝑡 × 𝜕𝑒𝑛𝑑𝑎𝑡𝑡 𝜕𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 = 0.292 × 0.152 = 0.044, compared with the direct effect of 0.178 for macro. 18 We include performance in introductory economics as a predictor of retention, but we do not use attitude in this equation. The main reasons for omitting attitude are that (a) it is statistically insignificant when included; (b) its inclusion reduces the sample size for this equation from 268 to 204 due to a low response rate; and (c) its omission does not affect the remaining coefficients within that smaller subsample, but its inclusion allows more precise estimation of the remaining coefficients due to the larger sample size. 36 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Table 4 Dependent Variable: Retention Score (1) (2) (3) (4) VARIABLES score_ret score_ret score_ret score_ret gp_mean 0.030* 0.030* 0.037 0.050 (0.016) (0.015) (0.037) (0.035) cumgpa 0.006 0.007 -0.008 -0.005 (0.026) (0.026) (0.056) (0.053) zactsat_math 0.003 0.001 0.015 0.001 (0.012) (0.012) (0.024) (0.024) zactsat_verbal 0.049*** 0.050*** 0.039* 0.040* (0.011) (0.011) (0.023) (0.021) male 0.014 0.018 0.020 0.046 (0.017) (0.017) (0.036) (0.035) minority 0.027 0.029 0.069 0.101 (0.033) (0.033) (0.065) (0.063) age 0.010 0.009 -0.008 -0.018 (0.008) (0.008) (0.020) (0.019) dlsv 0.026 0.031 (0.060) (0.060) dlsa 0.007 0.012 0.035 0.043 (0.027) (0.027) (0.071) (0.067) dlsr 0.003 0.002 0.086 0.126 (0.034) (0.034) (0.093) (0.089) dlsk 0.027 0.028 0.033 0.035 (0.020) (0.020) (0.044) (0.041) sophomore 0.049 0.050* 0.170 0.193* (0.030) (0.030) (0.108) (0.102) junior 0.057* 0.063* 0.140 0.181* (0.032) (0.032) (0.109) (0.103) senior -0.023 -0.016 0.000 0.000 (0.081) (0.081) (0.000) (0.000) nlargeclasses 0.024 0.021 0.071* 0.074* (0.023) (0.023) (0.042) (0.039) ntreatments 0.035* 0.075*** 0.063 0.240*** (0.018) (0.027) (0.061) (0.087) recent 0.057 0.300** 0.177* 0.659*** (0.061) (0.136) (0.091) (0.196) ntreatments  recent -0.290** -0.524*** (0.145) (0.191) microfirst 0.034 0.047 (0.034) (0.033) Constant 0.073 0.051 0.188 0.106 (0.180) (0.179) (0.456) (0.430) Observations 268 268 68 68 R-squared 0.272 0.283 0.345 0.430 Note: See notes for Table 2. 37 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 experiments, the annual rate of decline is 0.3 percentage points.19 This difference is both large and statistically significant, and is in addition to the indirect effect that experiments have on retention through their positive impact on performance in the introductory courses.20 A natural question is whether retention might be affected by the order in which the principles classes are taken. To examine this question we repeat this analysis using the subset of 68 students for which we have information on the quarters in which they took micro and macro. These are students who were in either a treatment or control section of both micro and macro that were part of this study. The variable microfirst indicates that the date of the micro course was earlier, which is the case for 43 of these 68 students. Results are reported in columns 3 and 4 of Table 4. As a likely consequence of the small sample size, all coefficients in column 3 other than the verbal score on the college entrance exam are statistically insignificant at the 10% level. However, when the recent variable is interacted with the number of treatment sections taken in column 4, the coefficients on ntreatments, recent, and their interaction are all statistically significant as in column 2. Given the larger sample size in column 2, however, we do not calculate depreciation rates for this small subsample. Conclusion The possibility of differing educational production functions for the introductory micro and macroeconomics courses sheds light on contradictory findings in the literature regarding the role of attitude and other factors in determining performance or retention in economics. We find that the determinants of, and the relationship between, attitude and performance differ across the two courses. Students master the concepts of and form attitudes toward economics differently in micro and macro. While it is not surprising that the economic education production functions differ across the introductory courses, this issue has not explicitly been explored previously. Our results suggest that the course itself may be an important determinant of the form of the production function, and the fact that previous studies use data from various economics courses at various levels may in part explain the lack of consistent findings. One of the ways in which the production processes differ is in the impact of classroom experiments. They positively affect attitudes in micro but not in macro, and they affect performance through different pathways, suggesting that the learning challenges are different 19 To illustrate the calculation, the coefficient on recent is score/recent = 0.300. But recent = months –1 , where months is the number of months since the most recent introductory economics course. So: score / months = (score / recent) x ((recent) / (months)) = 0.300  (–months –2 ) = –0.300/months 2 . This is the marginal effect of months for the control group. Evaluated at the mean of months for that group (9.27), the monthly percentage depreciation rate is: score / months = –0.300/(9.27) 2 = –0.296/86.0133 = –0.00348. On an annual basis this is –0.00348 × 12 = –0.04181, or –4.18 percentage points. For the treatment group, the coefficient is 0.300 – 0.290 = 0.010 and the mean of months is 6.18, giving an annual depreciation rate of –0.30 percent. 20 A possible concern is that there may be bias due to self-selection into the retention sample resulting from the effects of classroom experiments in the prerequisite principles courses. This concern is mitigated by the fact that virtually all—260 of the 268 students in the finance course—had declared as business administration majors prior to enrolling in principles. Thus the decision to include the finance course in their plan of study had effectively been made prior to enrollment in micro and/or macro for all but eight students in the sample. Because the variable indicating a student’s major was recorded in the principles course and not in the finance course, it is possible that some of these eight students might have changed their majors to business following their experience in principles. However, results are virtually identical when these eight students are dropped from the sample. Details are available on request. 38 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 and/or that experiments serve different purposes in the two courses. Experiments directly impact performance in macro, while they impact performance indirectly through attitude in micro. While students “feel” better about macro when experiments are used, it is not the better attitude that improves their performance. Better performance is a direct effect of the participation itself. Also of interest is that the impact of gender differs across the two courses in a similar manner, and perhaps for some of the same reasons. Gender has a direct impact on performance in macroeconomics, but only an indirect impact through attitude in micro. Activities that improve attitudes toward economics in the introductory micro course are worth exploring, as are activities that promote hands-on, personalized interaction with the concepts in introductory macro. Additionally, we find that retention depends primarily on past performance and native ability, but is also positively affected by the use of experiments. Classroom experiments reduce the “depreciation rate” of economic knowledge. This alone may warrant their serious consideration for inclusion in both of the introductory courses. Given the differing economic education production functions for the two principles courses, further discussion is needed on how the various types of introductory courses should be organized and taught. Our findings indicate that it is not appropriate for instructors to view teaching methodologies for all principles courses as interchangeable. They suggest that instructors in introductory micro likely need to move beyond straightforward explanations of the concepts and traditional presentations of the material. Micro instructors may need to exert additional effort to make initially unfamiliar and abstract concepts more intuitive and the models more applicable. It is important that these instructors understand the role that attitudes play in the learning process, the challenges students face in grasping concepts that may be counterintuitive initially (which might be related to attitude), and what might be needed for meaningful retention of ideas. While we often think of experiments as a way of illustrating ideas and giving hands-on experience with economic concepts, their most important contribution in microeconomics may be engaging students in a different way — changing the way they think or feel about the field itself, rather than just the concepts. Such engagement appears to be critically important in micro, as students may start the course with poorer attitudes towards the subject and/or more apprehension. And engagement does not mean simply giving students more time to understand a graph or equation, nor necessarily trying to make the material more accessible by using fewer graphs and charts. It means helping students believe they can understand the material and helping them want to understand the material. In this way, instructors can help provide students with the motivation to make the investment that is necessary to master the challenging material. References Anderson, G, D. Benjamin, and M. A. Fuss. 1994. “The Determinants of Success in University Introductory Economics Courses.” The Journal of Economic Education, 25(2), Spring, 99- 119. Bach, G. L. and P. Saunders. 1965. “Economic Education: Aspirations and Achievements.” American Economic Review, 55, June, 329-356. Ball, D. L., Thames, M. H., and G. Phelps. 2008. “Content Knowledge for Teaching: What Makes it Special?” Journal of Teacher Education, 59(5), November/December, 389-407. Basmann, R. L. 1960. “On Finite Sample Distributions of Generalized Classical Linear Identifiability Test Statistics.” Journal of the American Statistical Association 55(292), December, 650-659. 39 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Benedict, M. E. and J. Hoag. 2002. “Who’s Afraid of Their Economics Classes? Why Are Students Apprehensive about Introductory Economics Courses? An Empirical Investigation.” The American Economist, 46(2), Fall, 31-44. Borg, M. O. and S. L. Shapiro. 1996. “Personality Type and Student Performance in Principle of Economics.” The Journal of Economic Education, 27(1), Winter, 3-25. Borg, M. O. and H. A. Stranahan. 2002. “Personality Type and Student Performance in Upper- Level Economics Courses: The Importance of Race and Gender. The Journal of Economic Education, 33(1), Winter, 3-14. Brock, G.J. 2011. “Student Attitudes and Knowledge Change in an Introductory College Economics Course.” The Georgia Social Studies Journal, 1(1), 45-61. Cardell, N. S., Fort, R., Joerding, W., Inaba, F., Lamoreaux, D., Rosenman, R., Stromsdorfer, E., and R. Bartlett. 1996. "Laboratory-Based Experimental and Demonstration Initiatives in Teaching Undergraduate Economics." American Economic Review Papers and Proceedings, 86(2), 454-464. Charkins, R. J., D. M. O’Toole, and J. N. Wetzel. 1985. “Linking Teacher and Student Learning Styles with Student Achievement and Attitudes.” The Journal of Economic Education, 16(2), Spring, 111-120. Chizmar, J. F. Jr. and B. J. McCarney. 1984. “An Evaluation of a “Trade-Offs” Implementation Using Canonical Estimation of Joint Educational Production Functions.” The Journal of Economic Education, 15(1), Winter, 11-20. Craig, E. D. and J. B. O’Neill. 1976. “The Predictability of Retention in Economics.” The Journal of Economic Education, 7(2), Spring, 92-94. Crowley, R. W. and D. A. Wilton. 1980. “Retention of First-Year Economic Principles.” The Canadian Journal of Economics, 13(1), February, 132-135. Dancer, D. M. 2003. “The Gender Issue Revisited: A Case Study of Student Performance in Economics and Econometrics.” Economic Analysis and Policy, 33(1), March, 73-89. Dickie, M. 2006. “Experimenting on Classroom Experiments: Do They Increase Learning in Introductory Microeconomics?” The Journal of Economic Education, 37, 267-288. Durham, Y, T. McKinnon, and C. Schulman. 2007. “Classroom Experiments: Not Just Fun and Games.” Economic Inquiry, 45(1), January, 162-178. Emerson, T. L. and B. A. Taylor. 2004. “Comparing Student Achievement across Experimental and Lecture-Oriented Sections of a Principles of Microeconomics Course.” Southern Economic Journal, 70(3), 672-693. Feiner, S. and B. Roberts. 1995. “Using Alternative Paradigms to Teach About Race and Gender: A Critical Thinking Approach to Introductory Economics.” American Economic Review, 85(2), 367-371. Ferber, M. A. 1995. “The Study of Economics: A Feminist Critique.” American Economic Review, 85(2), 357-361. Fleming, N. D., and C. C. Bonwell. 1998. VARK. Frank, B. 1997. “The Impact of Classroom Experiments on the Learning of Economics: An Empirical Investigation.” Economic Inquiry, 35(4), 763-769. Frank, R.H., B. S. Bernanke, K. L. Antonovics, and O. Heffetz. 2016. Principles of Micro Economics, Sixth Edition. New York, NY: McGraw-Hill Education. Gremmen, H. and J. Potters. 1997. “Assessing the Efficacy of Gaming in Economic Education.” The Journal of Economic Education, 28(4), 291-303. 40 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Hausman, J. A. 1978. “Specification Tests in Econometrics.” Econometrica, 46(6), November, 1251-1271. Hodgin, R. F. 1984. “Information Theory and Attitude Formation in Economic Education.” The Journal of Economic Education, 15(3), Summer, 191-196. Jensen, E. J. and A. L. Owen. 2001. “Pedagogy, Gender, and Interest in Economics.” The Journal of Economic Education, 32(4), 323-343. Kane, J. and L. Spizman. 1999. “Determinants of Student Retention of Microeconomics Principles Concepts.” SUNY-Oswego Economics Department Working Paper 1999-01, Karstensson, L. and R.K. Vedder. 1974. “A Note on Attitude as a Factor in Learning Economics.” The Journal of Economic Education, 5, 109-111. Kohen, A. I. and P. H. Kipps. 1979. “Factors Determining Student Retention of Economic Knowledge after Completing the Principles of Microeconomics Course.” The Journal of Economic Education, 10(2), Spring, 38-48. Lumsden, K. G. and A. Scott. 1987. “The Economics Student Reexamined: Male-Female Differences in Comprehension.” The Journal of Economic Education, 18(4), Autumn, 365- 375. MacDowell, M. A., P. R. Senn, and J. C. Soper. 1977. “Does Sex Really Matter?” The Journal of Economic Education, 9(1), Autumn, 28-33. Manahan, J. 1983. “An Educational Production Function for Principles of Economics.” The Journal of Economic Education, 14(2), 11-16. Robb, R. E. and A. L. Robb. 1999. “Gender and the Study of Economics: The Role of Gender of the Instructor.” The Journal of Economic Education, 30(1), 3-19. Saunders, P. and G. L. Bach. 1970. “Lasting Effects of Introductory Economics: An Exploratory Study.” The Journal of Economic Education, 1, Spring, 143-149. Shulman, L. S. 1986. “Knowledge and Teaching: Foundations of the New Reform.” Harvard Educational Review, 57, 1-22. Staiger, D., and J. H. Stock. 1997. “Instrumental Variables Regression With Weak Instruments.” Econometrica, 65(3), May, 557-586. Stock, J. H., J. H. Wright, and M. Yogo. 2002. “A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments.” Journal of Business and Economic Statistics, 20(4), October, 518–529. Terregrossa, R. A., F. Englander, and Z. Wang. 2009. “Why Learning Styles Matter for Student Achievement in College Economics.” Journal for Economic Educators, 9(1), Summer, 16- 31. Walstad, W. B. 2001. “Improving Assessment in University Economics.” Journal of Economic Education, 32, 281-94. Walstad, W. B. and S. Allgood. 1999. “What Do College Seniors Know About Economics?” American Economic Review, 89(2), 350-354. Wetzel, J. N., Potter, W .J., and D. M. O’Toole. 1982. “The Influence of Learning and Teaching Styles on Student Attitudes and Achievement in the Introductory Economics Course: A Case Study.” The Journal of Economic Education, 13(1), Winter, 33-39. Wu, D-M. 1973. “Alternative Tests of Independence Between Stochastic Regressors and Disturbances.” Econometrica, 41(4), July, 733-750. Yandell, D. 1999. “Effects of Integration and Classroom Experiments on Student Learning and Satisfaction.” Working Paper, University of San Diego. 41 | J O U R N A L F O R E C O N O M I C E D U C A T O R S , 1 8 ( 2 ) , 2 0 1 8 Ziegert, A.L. 2000. “The Role of Personality Temperament and Student Learning in Principles of Economics: Further Evidence.” The Journal of Economic Education, 31(4), Autumn, 307- 322.