Survey Results 60 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 60 The Use of PRS in Introductory Microeconomics: Some Evidence on Performance and Attendance Christopher N. Annala, Shuo Chen, Daniel R. Strang 1 ABSTRACT This paper uses a sample of 425 students from 4 large sections of Introductory Microeconomics during the period 2005 – 2007 to examine the impact of using the Personal Response System (PRS or Clickers) on class attendance and exam performance. The evidence suggests that the use of PRS has led to improved attendance. The exam scores are similar to classes that used online quizzes instead of the PRS. A survey of student attitudes towards the PRS indicates that the use of PRS helps with student learning and reinforcing important concepts. Based on the results of this study the authors believe that the PRS is a useful tool for all instructors, particularly those faced with large sections. Introduction This paper studies the use of the Personal Response System (PRS or Clickers) in large sections of Introductory Microeconomics. The use of PRS is another in a long line of innovations in the delivery of content and the assessment of student learning. The PRS is a logical extension of the technology available in many classrooms and lecture halls, including projection systems and computers. The PRS also presents the opportunity for real time assessment; the instructor can monitor student learning at any point in a lecture or class discussion. This ―instant feedback‖ enables the instructor to provide additional coverage of topics that appear to give students difficulty. In this paper we examine evidence on PRS in two important areas: attendance and performance. In addition to straightforward quantitative measures, we also present the results of an attitudinal survey that was administered to better understand the impact of PRS on students. The results indicate that the use of PRS has had two important impacts. First, the use of PRS has led to improved attendance. Second, through the survey, students indicated that the use of PRS helped them to focus on material they found difficult to reinforce important concepts. Based on these results we believe that the PRS is a useful tool in the toolbox of all instructors, particularly those faced with large sections. It helps to force the students to attend class and it removes, to some degree, the anonymity some students feel in large lectures. Literature Review There has been a growing use of technology in economic pedagogy in the past 10 years. A key issue is the effect of technology on student performance. Sosin, et al (2004) use the post- and pre-course scores on the Test of Understanding College Economics (TUCE) to compare the effects of different technologies including PowerPoint, email, courseware, and web browsing. They find that using extensive technology in class leads to a small, but significant, improvement 1 Christopher N. Annala, Associate Professor of Economics, SUNY Geneseo; Shuo Chen, Assistant Professor of Economics, SUNY Geneseo; Daniel R. Strang, Professor of Economics, SUNY Geneseo. 61 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 61 in student performance. Interestingly, they find that individual technologies have different effects. For example, PowerPoint has a negative effect on student performance, courseware has a positive effect only in macro courses, and e-mailing materials has a positive effect only in micro courses. Agarwal and Day (1998) use both TUCE scores and final grades to analyze the impact of Internet tools, such as email, class mailing list discussion, and web pages on economic education. They find a positive correlation between use of Internet tools and exam grades. Manning (1996) reports that e-mail improved students’ communication with the instructor and with other students in her economics classes. As Goffe and Sosin (2005) point out, there is a trend toward using more technological innovations in economic teaching. The Personal Response System (PRS) is one of the new technologies. A number of instructors report that the PRS enhances teaching and learning experiences (e.g., Wood, 2004; Briggs, 2006; Elliott, 2003). The existing literature reports that using the PRS enhances student-instructor interaction and student concentration in class. Siegel (2004) also reports that using a technology similar to the PRS in an undergraduate architectural engineering class increases class attendance, when 5 percent of the final grade is associated with student participation. Hoffman and Goodwin (2006) employ a PRS in teaching library literacy at Texas A & M University, finding that the clickers make the classroom more interactive and learner-centered. Several studies find a correlation between attendance and learning in undergraduate economic education. Romer (1993) finds a correlation between attendance and performance after controlling for GPA. Earlier studies report similar results (e.g. Schmidt, 1983; Park and Kerr, 1990). Romer (1993) also points out that attendance is not exogenous and so the effects of omitted variables cannot be singled out. Two recent studies (Cohn and Johnson, 2006; Stanca, 2006) present evidence that attendance has a significant effect on learning after controlling for an array of variables that reflect student heterogeneity in demographics, ability, effort, and motivation. Marburger (2006) experiments with enforcing a mandatory attendance policy and finds significantly reduced absenteeism and improved exam performance. Hence, if using the PRS effectively increases attendance, it should be expected to increase exam performance. Although there are a number of descriptive studies on using the PRS, there are few quantitative analyses of the impact of using the PRS on student performance. Two recent studies have sought to provide a detailed quantitative analysis of the link between PRS and performance. First, Ball et al (2005) study a wireless interactive teaching system (WITS) used in Principles of Economics classes. Using the system significantly increased final grades compared to a control class. Further, women and freshmen realized the largest improvement from using the system. In Ball, et al (2005), however, the students use the handheld device only when acting as economic agents in classroom games. It is not surprising that students get a better understanding of the games by participating in the games instead of just reading the game results. We add to this evidence by collecting data on a PRS used as a formal tool of assessment. We then estimate a regression equation to study the impact of PRS on final exam grades. The study most closely related to ours is by Carnaghan and Webb (2007). Here the authors use ―Group Response Systems‖ in a management accounting course. These authors study issues similar to ours in that they perform an attitudinal survey and estimate the impact that PRS has on exam scores. The setup of the Carnaghan and Webb study differs from ours, however, but there are a number of similarities and similar conclusions. Students state that the PRS was useful in 62 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 62 learning/reinforcing the material, but there was little impact on exam performance from the use of PRS. These issues are discussed in greater detail below. Methodology This study utilizes information on students in four sections of Principles of Microeconomics gathered over a two year period. The four sections include a section of 110 students in the fall of 2005, a section of 113 students in the fall of 2006, and two sections in the spring of 2007, with one section at 100 students and the other at 102. The same professor served as the instructor for all four sections. One section used online quizzes (fall of 2005) in which the class was divided into three- person teams and the team members worked together on the quizzes. The quiz sessions were conducted by the three-person teams outside the classroom and no restrictions were imposed on resources used by students during quiz sessions. Additionally, the three-person teams were allowed to make multiple attempts at the quizzes and to submit the highest quiz scores for inclusion in the course grade. In contrast, the clicker sections administered quizzes during the lecture periods. Students were encouraged to confer on their answers and were awarded points for the answers based on the following scheme: 2 points for each correct answer, 1 point for each incorrect answer, and 0 points for failure to answer. Duncan (2005) recommends this point allocation scheme, suggesting that, ―Many instructors have the goal of increasing participation through clicker use. One way to do this is to give partial credit for wrong clicker answers.‖ Anecdotally, the students seemed to like the award system. Students were encouraged to confer for two reasons—to promote the synergies of students working together, and more pragmatically, to avoid the enforcement of a no-collaboration restriction. A notable difference between the online quizzes and the clicker quizzes was that the online approach allowed students a virtually unlimited number of quiz attempts. In the fall of 2006, the instructor initiated the use of a PRS to instantaneously record student responses to quiz questions. Each student was required to have a PRS unit (clicker) and responded to quiz questions that the professor administered during the lecture sessions. The students were informed that 20% of their course grade was based upon their performance on ―clicker quizzes.‖ The class was taught at 8:30 a.m. on Tuesdays and Thursdays. Quizzes were administered on most days during which a major exam was not being administered or discussed. The quizzes were administered at various points during the class sessions--beginning, middle and end. On some occasions, two quizzes were given during a single class session. Table 1 describes the variables used in the experimental design. Although clickers were not used in the fall of 2005, many of the other potentially significant factors were the same as those for the fall of 2006 section. The classes were taught at 8:30 a.m. on Tuesdays and Thursdays, the number of students was virtually same (110 versus 113) and testing procedures, with the exception of the quizzes, were the same. There is no reason to believe that the composition of students in terms of major/non-major and class year was different in any significant way. Table 1 provides comparison data between the two cohorts from the Fall of 2005 and Fall of 2006. These data come directly from the Student Opinion of 63 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 63 Table 1: Experimental Design Variable Section Fall 2005 Fall 2006 Spring 2007 Spring 2007 Time T TH 8:30 am T TH 8:30 am T TH 8:30 am T TH 11:20 am Instructor Prof. XXX Prof. XXX Prof. XXX Prof. XXX Class Size 110 113 100 102 Gender 55% Male 45% Female 46% Male 53% Female 48% Male 51% Female 55% Male 44% Female Required Course 83% Yes 17% No 87% Yes 13% No 52% Yes 48% No 69% Yes 31% No Class Standing 48% Freshman 31% Sophomore 14% Junior 5% Senior 40% Freshman 42% Sophomore 12% Junior 4% Senior 46% Freshman 31% Sophomore 9% Junior 13% Senior 58% Freshman 25% Sophomore 11% Junior 4% Senior Expected Grade (Student Response) 31% A 52% B 14% C 34% A 54% B 8% C 44% A 40% B 18% C 32% A 46% B 17% C Hours Spent Studying Outside of Class 35% 0-2 49% 3-5 11% 6-8 37% 0-2 48% 3-5 6% 6-8 37% 0-2 48% 3-5 14% 6-8 27% 0-2 53% 3-5 16% 6-8 Quiz Method On-line PRS PRS PRS Quiz Weight in Final Grade 20% 20% 20% 20% Teamwork Permitted Yes Yes Yes Yes Textbook Microeconomics, McConnell and Brue Microeconomics, McConnell and Brue Microeconomics, McConnell and Brue Microeconomics, McConnell and Brue Final Exam Version A Version A Version B Version C 64 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 64 Faculty Instruction (SOFI) surveys which are completed by the students. 2 In addition to the demographic similarities, the same textbook was used in both sections. Thus, the only major difference was that students in the Fall 2005 section received 20% of their grade from quizzes provided by the textbook publisher that they took online, whereas students in the Fall 2006 section received 20% of their grade from clicker quizzes. The use of clickers offers the advantage of ease of maintenance of attendance records. Attendance information was recorded for the three sections of Microeconomics in fall 2006 and spring 2007. It was not practical to collect attendance data for the Fall 2005 section. The grades for all four sections were based on the same formula: 20% of the grade was based upon quizzes, 50% on three preliminary exams spaced out throughout the semester, and 30% from a comprehensive final. All of the preliminary exams and the final were multiple choice exams. In order to a measure of the impact of clicker quizzes, the same final examination was administered to the all sections. Given that virtually every other key variable except for the use of clickers was the same across the sections, differences in the final examination results are attributed to the impact of the clicker quizzes versus online quizzes. Analysis In examining our preliminary evidence, we found that the use of the PRS seemed to have positive effects on attendance (Table 2). Romer (1993), for example, reported an average attendance rate of 67 percent in undergraduate economics classes at three major universities. He also reported higher absenteeism in large classes and Principles courses. In comparison, the attendance rates in our classes using the PRS were 88% in Fall 2006, 87% in the Spring 2007 8:30 a.m. section, and 89% in the Spring 2007 11:20 a.m. section. Moreover, in the three PRS classes, thirty-six percent of the students had perfect attendance; 60 percent missed one class; and 75 percent missed none or fewer than two classes (Table 3). Our university does not allow attendance to count as part of the student grade, so few if any faculty take attendance, especially in classes of this size. Nevertheless, we do have self- reported results from the student surveys in both the fall of 2005 and fall of 2006. These results show a dramatic increase in the number of students responding ―Always Went‖ to the question, ―How was your attendance?‖ In the fall of 2005, 48 percent of students indicated that they ―Always Went‖ compared to 75 percent of students in fall of 2006. 3 Although one might question the reliability of self-reported attendance results, the top two categories (4 and 5, with 5 translating to Always Went) were selected by 91% of the students responding in the fall of 2006 and this is consistent with the attendance numbers actually recorded via the clickers (Table 4). The strong relationship between self-reported attendance and clicker attendance seems to validate the use of self-reported attendance. In the fall of 2005, the two top categories of self-reported attendance (i.e. 4 and 5) were selected by 79% (48% + 31%) of the students responding. Simple triangulation of these results suggests the actual attendance 2 A slight complication may arise when comparing student surveys, because our school converted from an in-class paper and pencil op-scan survey to an on-line survey in Spring 2006. We believe that this change has only a marginal effect on the comparison, but felt that it should be noted. 3 There is a difference between the self-reported attendance from the Student Opinion Survey and our estimate, due to the fact that in 2006 the new student survey is on-line and not all students filled out the on-line survey. 65 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 65 during this semester was significantly lower than the 88% attendance regularly observed in subsequent semesters using clicker attendance information. Table 2: Attendance rates (Based on PRS response) Class Attendance Fall 06 88% Spring 07 8:30 section 87% Spring 07 11:20 section 89% Table 3: Percentage of students who missed classes Number of Missed Classes Percentage of Students Cumulative Percentage of Students 0 36.0 36.0 1 23.7 59.7 2 14.9 74.6 3 9.4 84.0 4 4.2 88.2 5 2.9 91.1 6 5.2 96.3 7 1.0 97.3 8 or more 2.6 100.0 Table 4: Self-Reported Attendance Information from Student Surveys. Students respond using a spectrum (1 to 5) to the question: How was your attendance? Attendance Fall 2005 No PRS Fall 2006 With PRS 1. Missed 50%+ 1% 0% 2. 4% 0% 3. Missed 25% 16% 9% 4. 31% 16% 5. Always Went 48% 75% The evidence suggests that the use of PRS in the classroom had positive benefits in terms of attendance, but we are also interested in determining whether the use of the PRS is related to success in the course. In the survey, students were asked to respond to the statement; ―The results of my in-class quizzes using clickers and my exam scores were strongly related.‖ A total of 58 students, or 31.35% of students, indicated that they ―strongly agreed‖ or ―agreed‖ with the statement. To further study the relationship between performance on the in-class PRS quizzes and student performance on the final exam, we estimated the following regression equation: 66 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 66 i egradequizPRSgradeexamFinal )__(__ 10 The results of the regression estimation appear in Table 5. Based on the t-statistics, the results indicate that higher quiz grades are significantly associated with higher final exam scores. It must be noted that the relationship is relatively weak, given the generally low adjusted R 2 values. For comparison purposes, and as a test of robustness, we estimate the same regression equation using data from the previous semester, fall 2006. These results are similar to the spring 2007 results, where the coefficient on quiz grades is statistically significantly different from zero at conventional levels, yet the regression provides a relatively poor fit as demonstrated by an adjusted R 2 of 0.175. Table 5: Estimation results: Dependent variable is final exam score. (t-statistics in parentheses) Spring 2007 8:30 Section Spring 2007 11:20 Section Spring 2007 Combined Sections Fall 2006 Combined Sections Constant 37.896 (5.45) 52.305 (6.52) 44.683 (8.52) 41.738 (6.483) Quiz Grade 0.216 (5.23) 0.045 (2.38) 0.162 (5.36) 0.184 (4.98) Adjusted R 2 .212 .044 .122 .175 Observations 98 101 199 112 Another measure of the impact of clickers utilized the results on the final examination for students in the sections for the fall of 2005 and the fall of 2006. The same multiple choice exam (Version A in Table 1) was administered to both sections. Since most other key variables were controlled, including instructor, time of day, class size, instructional format, and student audience, any difference in results on the final could be attributed to clickers. The results are presented in Table 6. Table 6. Difference in means of exam grades Class Section Number of Students Quiz Method Mean grade on final exam Standard deviation Fall 2005 110 Online 74.7 16.0 Fall 2006 113 PRS 73.4 14.4 A simple test of difference in means indicates that there is no significant difference in the means for the two finals. One might expect that if clicker quizzes represent a superior form of pedagogy, then the mean grade on the final exam for the clickers section (Fall 2006) would 67 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 67 exceed that for the section using online quizzes (Fall 2005). Initially this absence of a significant difference in performance was perplexing. On further consideration, however, it might be understandable. In both treatments students were given the opportunity to test their learning by taking quizzes. In the case of the online quizzes, students were encouraged to take multiple quizzes to review each topic. This was not the case for the clicker quizzes. So, in both cases students had the opportunity to test their knowledge using quizzes. If there is value in both online quizzes taken by students outside the classroom and clicker quizzes taken during class sessions, perhaps some combination of the two modalities would be closer to optimal. This presents an opportunity for future research on the effect of PRS on student performance. Although the impact of PRS use on grade performance is not a compelling reason for an instructor to adopt PRS for assessment, there are other potential benefits to the use of PRS. Through the use of a survey we attempt to identify some of these encouraging results. Survey Results A survey was developed by the authors to ascertain the student’s perceptions and attitudes regarding the use of PRS. The survey consisted of 16 questions, including both demographic and attitudinal questions. The survey was administered in the two Introductory Microeconomics classes in Spring 2007, taught by the same professor, in back-to-back time slots, with a combined total of 195 students. Ten (10) students indicated Economics as their major (or double major), an additional 50 students indicated a major of Business Administration, and 18 students identified their major as Accounting. These students account for 28.25% of the total and represent students within the School of Business. A total of 119 students (61.66%) indicated that Introductory Microeconomics fulfills a major requirement for their particular program. As this is an introductory course, 81.54% of students identified their class level as either freshman or sophomore. Table 7: Sample by major, fall 2007 Major Count Economics 10 Business Administration 50 Accounting 18 Other 117 Total 195 We also asked students to indicate their GPA based on quartiles developed from previous semesters. These results are presented in Table 8. 46.39% of students stated that their GPA was 3.04 or greater and 18.56% of students indicated a GPA less than 2.68. A category was also included for students who may not have a College YYY GPA, as they may be transfer students or spring semester enrollees. To gain a better understanding of the group, students were asked two questions regarding their experience with PRS. The first question asked whether this was the first class in which they had used the PRS, to which 172 or 89.12% answered yes. Twenty-one (21) or 10.88% of students indicated prior experience with PRS. The students were also asked if the PRS was easy to operate; 94% of students either ―strongly agreed‖ or ―agreed‖ (Table 9). This indicates that 68 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 68 potential technical issues should not bias our results and that students were comfortable with the technology. Table 8: My current GPA falls into which category, fall 2007 GPA Count Below 2.68 36 From 2.68 to 3.03 53 From 3.04 to 3.51 55 Above 3.51 35 I don’t currently have a College YYY GPA 15 Total 194 Table 9: Clickers (PRS) were easy to operate, fall 2007 Response Option Count Strongly Agree 98 Agree 85 No Opinion 3 Disagree 6 Strongly Disagree 1 Total 193 One of our most important questions about the use of PRS was whether or not daily, or near-daily, PRS quizzes, given at various times during the class, (beginning, middle, or end of class) and frequently given twice during a class, would have a positive effect on attendance. Increased attendance appears to improve classroom performance (Stanca, 2006; Marburger, 2006). The use of the PRS for quizzing was designed to provide a strong incentive for students to attend class. Table 10 presents the results when students were asked to provide an approximation of their attendance; 90.21% of students indicated that their attendance exceeded 81% of classes. These results match closely with the other attendance measures presented in Tables 2 through 4. On a related issue, we also asked students whether the use of PRS increased their attendance in the class, to which 85.57% of students either ―agreed‖ or ―strongly agreed,‖ (Table 11). 4 Several of the remaining questions focused on the relationship between the use of the PRS and class performance. These questions included (1) whether the use of PRS quizzes increased focus during class, (2) whether the quizzes helped students identify what they needed to study, (3) whether the PRS quizzes helped students better remember the material covered 4 Based on casual observation, the authors believe that the student responses represent the truth, as attendance appeared to be much higher with the use of PRS than in previous semesters. Furthermore, as demonstrated by the comparison to fall 2005, these results appear to have been borne out empirically. 69 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 69 during lecture, and (4) whether the PRS quizzes were helpful in reinforcing course material. The results of these questions appear in Table 12. Table 10: My approximate class attendance was, fall 2007 Response Option Count 91 percent or more 161 81 to 90 percent 14 71 to 80 percent 13 61 to 70 percent 4 60 percent or less 2 Total 194 Table 11: The use of clickers (PRS) in this class increased my attendance in this class, fall 2007 Response Option Count Strongly Agree 114 Agree 52 No Opinion 14 Disagree 13 Strongly Disagree 1 Total 194 Table 12. Additional items surveyed, fall 2007 Response Option Question Strongly Agree Agree No Opinion Disagree Strongly Disagree Total Focus 54 85 22 29 4 194 What to Study 32 115 21 16 1 185 Remember 24 98 33 29 1 185 Reinforcement 49 103 21 11 1 185 The student responses indicate that the PRS was an important tool in identifying the material that required additional study, as 79.46% of students selected ―strongly agree‖ or ―agree‖ to the statement: ―The in-class quizzes using clickers gave me ideas about what I needed to study.‖ Additionally, students also believed that the use of the PRS quizzes were helpful in reinforcing the course material, as 82.16% of students either ―strongly agreed‖ or ―agreed‖ with the statement: ―The in-class quizzes using clickers were effective in reinforcing the course material.‖ We believe that these results, combined with the evidence related to attendance by Romer (1993), represent important information in better understanding how to present and assess the large amount of material covered in a typical introductory economics course. Based on the survey results, the use of the PRS appears to increase student attendance. Once the students are 70 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 70 in the classroom, the use of the PRS enables real time assessment, which helps students not only remember the material, but also to identify areas which presented problems for them. Concluding Remarks We present results related to the use of PRS in large sections of Introductory Microeconomics. The empirical evidence indicates that the use of PRS has positive effects on student attendance. Based on our survey, we also find that students generally had a positive response to the use of PRS for real time assessment. Unfortunately, the results of our simple quantitative analysis were not particularly strong. This may be the result of model misspecification, including omitted variable bias. It may also be the case that the use of PRS represents a mediating variable between attendance and performance. Future research may better reveal the link between the use of PRS and student performance. Nevertheless, we believe that our results indicate that PRS may be a useful tool in increasing student attendance, which according to the existing research is closely related to performance. The use of PRS can be a valuable instrument for instructors desiring to improve attendance and to better monitor students’ performance in real time. References Agarwal, R., and A. E. Day. 1998. ―The Impact of the Internet on Economic Education.‖ Journal of Economic Education, 29: 99 – 110. Ball, S. B., C. C. Eckel, and C. Rojas. 2005. ―Technology Improves Learning in Large Principles of Economics Classes: Using Our WITS.‖ Presentation at the American Economic Association meeting, San Diego, January. Briggs, L. L. 2006. ―Response Devices Keep FSU Students Focused.‖ http://campustechnology.com/articles/41307/, accessed on 3/16/2007. Carnaghan, C., and A. Webb. 2007. ―Investigating the Effects of Group Response Systems on Student Satisfaction, Learning, and Engagement in Accounting Education.‖ Issues in Accounting Education, 22: 391-409. Cohn, E., and E. Johnson. 2006. ―Class Attendance and Performance in Principles of Economics.‖ Education Economics, 14: 211 – 233. Duncan. 2005. Clickers in the Classroom. Pearson-Addison Wesley, San Francisco. Elliott, C. 2003. ―Using a Personal Response System in Economics Teaching.‖ International Review of Economics Education, 1: 80 – 86. Goffe, W. L., and K. Sosin. 2005. ―Teaching with Technology: May You Live in Interesting Times.‖ Journal of Economic Education, 36: 278 – 291. Hoffman, C. and S. Goodwin. 2006. ―A clicker for your thoughts: technology for active learning.‖ New Library World , 107: 422. Manning, L. M. 1996. ―Economics on the Internet: Electronic Mail in the Classroom.‖ Journal of Economic Education, 27: 201 – 204. Marburger, D. R. 2006. ―Does Mandatory Attendance Improve Student Performance?‖ Journal of Economic Education, 37: 148 -155. Park, K. H., and P. M. Kerr. 1990. ―Determinants of Academic Performance: A Multinomial Logit Approach.‖ Journal of Economic Education, 21: 101 – 111. Romer, D. 1993. ―Do Students Go to Class? Should They?‖ Journal of Economic Perspectives, 7: 167 – 174. http://campustechnology.com/articles/41307/ 71 JOURNAL FOR ECONOMIC EDUCATORS, 8(2), FALL 2008 71 Schmidt, R. M. 1983. ―Who Maximizes What? A Study in Student Time Allocation.‖ The American Economic Review, 73: 23 – 28. Siegel, J. A. 2004. ―INTICE – Interactive Technology to Improve the Classroom Experience.‖ http://www.ph.utexas.edu/~ctalk/bulletin/intice.htm, accessed 6/16/2007. Sosin, K., B. J. Blecha, R. Agarwal, R. L. Bartlett, and J. I. Daniel. 2004. ―Efficiency in the Use of Technology in Economic Education: Some Preliminary Results.‖ The American Economic Review, 94: 253 – 258. Stanca, L. 2006. ―The Effects of Attendance on Academic Performance: Panel Data Evidence for Introductory Microeconomics.‖ Journal of Economic Education, 37: 251 – 266. Wood, William. 2004. ―Clickers: A Teaching Gimmick that Works.‖ Developmental Cell, 7:796-98. http://www.ph.utexas.edu/~ctalk/bulletin/intice.htm