Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 88 Do continuous assessment results affect final exam outcomes? Evidence from a microeconomics course Juan Carlos Reboredo Dpto. de Fundamentos da Análise Económica. Universidade de Santiago de Compostela, Avda. Xoán XXIII, s/n, 15782 Santiago de Compostela, Spain. * Corresponding author: Email: juancarlos.reboredo@usc.es; Phone: + 34 881811675 Received: 2016-01-09; Accepted: 2016-07-21 Abstract Continuous assessment aims to enhance student learning and understanding of a subject and so achieve better educational outcomes. We investigated how continuous assessment grades affected final exam grades. Using a dataset for six academic post-Bologna Process years (2009-2015) for a first-year undergraduate microeconomics course offered at a Spanish public university, we examined conditional dependence between continuous assessment and final exam grades. Our results would indicate a limited contribution of continuous assessment results to final exam results: the probability of the final exam performance improving on the continuous assessment grade was lower than the probability of the opposite occurring. A consistent exception, however, was students who obtained an A grade for continuous assessment. Our results would cast some doubt on the beneficial effects of continuous assessment advocated by the Bologna Process. Keywords Continuous assessment; final exam performance; conditional dependence Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 89 1. Introduction Implementation of the European Higher Education Area (Bologna Process) led to the introduction of continuous assessment as a way to enhance and measure student learning. Traditional lectures and final examinations have therefore been replaced by expositive and practical classes combined with student self-learning activities that are continuously assessed. A student’s final grade is comprised of marks for continuous assessment and for a final global exam. Grades therefore depend not only on the ability of the lecturer but also on the student’s own motivation, dedication and abilities. Examining the impact of continuous assessment on final student performance has several potentially important implications: for the efficient allocation of scarce resources in higher education institutions, for student time management aimed at maximizing academic performance and for the Bologna Process aim of enhancing student learning and understanding. Despite the fact that continuous assessment is a core principle underlying the Bologna Process, however, evidence on the impact of continuous assessment on final exam grades for different subjects is still lacking in most EU countries. For a microeconomics undergraduate course, we investigate the relationship between continuous assessment and final exam grades by examining the conditional dependence between these two kinds of grades. We considered four different grades (A, B, C and D) for both kinds of assessment and computed the probabilities of each final grade conditional on specific continuous assessment grades. We formulate three hypotheses regarding the positive, null or negative contribution of continuous assessment grades to final exam grades and tested the hypotheses using a likelihood ratio test for conditional dependence. We applied our modelling procedure to a sample of students enrolled in the same first-year undergraduate microeconomics course over six years (2009-2015). Our empirical evidence would indicate that continuous assessment grades made a limited contribution to final exam grades: the conditional probability of achieving a better grade in the final exam compared to continuous assessment was significantly lower than the conditional probability of getting Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 90 a lower grade. However, for students who obtained a grade A in continuous assessment, the conditional probability of obtaining an A in the final exam was no lower than the conditional probability of getting a lower grade. Overall, however, final exam grades were generally lower than the grades that would be expected on the basis of continuous assessment outcomes. Our results for the analysed microeconomics courses would suggest that continuous assessment aimed at fostering student learning and understanding may have neutral or even detrimental effects on student learning outcomes. Poorer performance in final exams is likely due to students not fully assimilating a subject, which, in turn, may be due to rational management of scarce study time aimed at maximizing the probability of a pass on the basis of continuous assessment and dedicating more time to subjects for which they may obtain better marks. Our results have implications for the efficiency of resource allocation to more personalized learning aimed at improving overall educational outcomes (e.g., the greater cost of reduced-size classes), which are highly dependent on student motivation and choices. The remainder of this article is organized as follows: Section 2 reviews relevant literature; Section 3 describes the data; Section 4 outlines our methodological approach; Section 5 presents and discusses the results; and, finally, Section 6 concludes. 2. Brief literature review Previous literature on student performance that examines the relationship between lecture attendance and exam performance and between study time and student grades reports mixed evidence for the impact of attendance and study time on academic performance. A seminal study by Romer (1993), based on attendance records for an intermediate-level macroeconomics course, reported attendance to have a positive and significant impact on academic performance. This conclusion, which has been corroborated by other empirical studies (see, e.g., Durden and Ellis, 1995; Devadoss and Foltz, 1996; Chan et al., 1997; Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 91 Rodgers, 2002; Kirby and McElroy, 2003; Dolton et al., 2003), would suggest that mandatory attendance policies enhance student performance. However, other studies have found otherwise, namely, that incentives to attendance have no impact on academic performance (Chen and Lin, 2015; Rodgers, 2002). Yet other studies have found that the positive link between attendance and performance may be explained by endogeneity problems. Krohn and O’Connor (2005), for a study of macroeconomics courses, found no relationship between attendance and grades when using instrumental variables to account for endogeneity. Similarly, Martin and Walker (2006), using fixed effect estimators to account for endogeneity, found that the positive and significant effect of class attendance disappeared when using panel data estimations instead of ordinary least squares (OLS). More recently, Andrietti (2014) also reported similar findings for a panel-data study. Another strand of the literature has examined the impact of study time on student grades. Several studies have reported no significant impact of study time on grades (Schuman et al., 1985; Hill, 1991; Rau and Durand, 2000) and Plant et al (2005) found that the amount of study by college students was a poor predictor of academic performance. In contrast, in a study that considered causal effects, Stinebrickner (2008) found that study time mattered, reporting that each additional study hour increased grades by 0.36 points. Similarly, Bonesrønning and Opstad (2012) found that increased study effort improved test grades. In contrast with the existing literature, we do not consider attendance or study time but, instead, outcomes for both these variables as reflected in continuous assessment grades. Continuous assessment provides useful information to both teachers and students, as teachers can identify the main areas of difficulty and students can pinpoint lack of sufficient knowledge, allowing an opportunity for both to rectify. Continuous assessment can therefore act as an early-warning system that alerts both teachers and students to the need to refocus their respective efforts in the light of results. On that basis, if the final exam covers all the topics reflected in the continuous assessment tests, its outcome will be no worse than the outcome for the continuous assessment. This is the spirit underlying Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 92 continuous assessment of student learning as introduced by the Bologna Process. We test for this hypothesis by considering the conditional dependence of final exam grades on continuous assessment results. 3. Data We collected data from assessments for students taking a first-year microeconomics course taught in the spring semester of a business administration undergraduate degree offered at a Spanish public university (Universidade de Santiago de Compostela). Course content included analyses of consumption and production decisions and competitive market functioning. The course was delivered in a two-hour theoretical lecture plus a 90-minute reduced-size, interactive practical session per week. Our database of 32 hours of theoretical classes and 17 hours of interactive classes contained data for six academic years (2009- 2015) subsequent to launch of the Bologna Process. Students were evaluated using scored tests and exercises, weighted in a similar way and with the same level of difficulty as the final exam (which students had to take at the end of the course) consisting of theoretical and practical content. The course was taught each year by the same teacher using the same syllabus and the same assessment procedures. Grading was as follows: A, 90% or more; B, 70%-89%; C, 50%-69%; and D, 49% or less. Table 1 reports descriptive statistics for our data referring to 589 students. The sample reflected academic performance data unevenly distributed over the sampling years. The percentages for students obtaining each grade point to a variation in final exam grade distribution with respect to continuous assessment grade distribution. Thus, largely the same proportions of students obtained an A in both kinds of assessments (except for 2009- 2010); the share of students who obtained a B-grade for continuous assessment and who maintained this grade in the final exam, however, was greatly reduced; and the share of C- and D-grade students receiving the same grades in the final exam increased. In other words, Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 93 the probability distribution for continuous assessment grades moved left: the probability of an A remained similar, the probability of a B dropped and the probability of a C or D rose. This descriptive evidence would indicate that continuous assessment yields confusing evidence of future student performance: the hard-working and high-performing students (A grade) performed consistently, obtaining the same top grades in both assessments; students with reasonably good continuous assessment results (B grade) would seem to have slackened off their efforts and, consequently, obtained poorer final exam grades; and students with poor continuous assessment results (C and D grades) would seem to have invested lower effort and, consequently, do not improved their final exam grades. The descriptive statistics in Table 1 also show that student performance worsened with class size: the larger the class, the greater the leftward movement of the final exam grade distribution with respect to the continuous assessment grade distribution (see, for instance, results for the 46 students of 2014-2015 compared to results for the 194 students of 2010- 2011). Table 1. Descriptive statistics. Continuous assessment grade vs final exam grade. Continuous assessment grade (% students) Final exam grade (% students) Year # Students A B C D A B C D 2009-2010 60 13.5 47.5 25.4 13.6 6.8 16.9 52.6 23.7 2010-2011 194 0.5 17.1 46.6 35.8 2.6 11.4 30.6 55.4 2011-2012 80 3.7 7.5 30.1 58.7 10 22.5 28.7 38.8 2012-2013 139 2.9 4.4 47.8 44.9 0.7 3.6 52.9 42.8 2013-2014 70 5.7 34.3 22.9 37.1 5.7 12.9 14.3 67.1 2014-2015 46 13.0 39.2 13.0 34.8 10.9 13.0 34.8 41.3 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 94 4. Modelling conditional dependence We modelled continuous assessment and final exam grades to test for conditional dependence between grades as follows. Consider indicators for the grades that student i can obtain for the final exam (fe) and for continuous assessment (ca), fe i I and ca i I , respectively, as taking the values j=A, B, C or D. Conditional dependence of the final exam grade on the continuous assessment grade for each student is given by  fe cai iPr I h | I j  , for h,j=A, B, C, D, and conditional dependence implies that    fe ca fei i iPr I h | I j Pr I h    . Conditional dependence between grades can thus be represented by the conditional dependence matrix P: AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD P P P P P P P P P P P P P P P P P               , (1) where each element, given by  fe cajh i iP Pr I h | I j   for h,j=A, B, C, D, reports the probability of a grade h in the final exam provided the continuous assessment grade was j. Conditional probability estimates are obtained by maximizing the maximum likelihood function under conditional dependence, given by:   jh n jh j,h P P  , (2) where jhn is the number of students with continuous assessment grade ca i I j followed by final exam grade fe i I h . Estimated parameters arising from Eq. (2) are simply the ratio of the counts for the corresponding cells: jh jh jA jB jC jD n P̂ n n n n     , (3) Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 95 In light of the independence tests proposed by Christoffersen (1998) and Reboredo (2014) for indication sequences, hypotheses regarding the impact of the continuous assessment grade on final exam performance can be formulated as specific restrictions on the conditional probabilities inside matrix P in Eq. (1). These restrictions can be tested using standard likelihood ratios tests. We consider three hypotheses. First, we consider the hypothesis that final exam grade is independent of the continuous assessment grade (H1), formulated as:  Hypothesis 1: 0 jh hH :P P The conditional and unconditional probability of a specific grade h in the final exam are thus similar. Note that this hypothesis states that the conditional probabilities in the h- column of the matrix P in Eq. (1) are equal. Taking the likelihood function under the null hypothesis 1, we can obtain the likelihood ratio test as: jhh nn h jh h j 2log P P           LR . (4) A second hypothesis consists of testing whether the final exam grade is better than the continuous assessment grade. In this case, the conditional probability of getting a better final exam grade is greater than the conditional probability of obtaining the same or a lower continuous assessment grade. This hypothesis (H2) can be formulated as:  Hypothesis 2: 0 jh jzH :P P , where h is a better grade than j and z is a poorer grade than h. Using the likelihood function we can estimate the likelihood ratio for H2 as: jh jh n n jh jh h z h 2log P P           LR . (5) Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 96 Finally, the third hypothesis is that continuous assessment and final exam grades are the same. Thus, the conditional probability of obtaining the same grade in the continuous assessment and final exam is greater than the conditional probability of obtaining a different grade. Accordingly, each conditional probability in the diagonal of matrix P will be greater than the conditional probabilities in the same row. This hypothesis (H3) can be formulated as:  Hypothesis 3: 0 hh hzH :P P , h z  Using the likelihood function under null H3, we obtain the likelihood ratio as: hh hz n n hz h z 2log P P          LR . (6) 5. Results Table 2 reports estimates for the conditional probability matrix P in Eq. (1) considering the full sample of student grades. The empirical results show that conditional dependence between continuous assessment and final exam performance varied according to grade (A, B, C or D). They also reveal substantial differences in conditional transition probabilities for the different grades. The implications for the effectiveness of continuous assessment in terms of final exam performance merit investigation, as, broadly speaking (and excluding the A grades), the conditional probabilities of getting a better final exam grade than continuous assessment grade were notably lower than the conditional probabilities of gaining a similar or lower grade. Table 2. Conditional probabilities. 0.39 0.27 0.30 0.04 0.09 0.30 0.49 0.12 P̂ 0.04 0.06 0.45 0.45 0.01 0.05 0.22 0.72               Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 97 We next tested the three hypotheses formulated in the previous section. First, we considered the hypothesis that final exam grade was independent of the continuous assessment grade (H1). The results (Table 3) indicate that the null of independence was rejected for different grades, indicating that final exam performance depended on continuous assessment outcomes. The key question, answered by H2 and H3, is what shape does the conditional dependence take and what is its contribution to final exam outcomes. Table 3. Test for hypothesis 1. 0 jA A H :P P 0 jB BH :P P 0 jC CH :P P 0 jD DH :P P Rejection Rejection Rejection Rejection H2 tested whether students improved in their continuous assessment grade in their final exam (Table 4). Our evidence indicates that A-grade students were likely to achieve the same grade in the final exam, whereas B-, C- and D-grade students were unable to achieve a better grade. Thus, the results of testing H2 would point to continuous assessment being of dubious value, as only the hardest-working students with excellent continuous assessment results performed equally as well in the final exam. Table 4. Test for hypothesis 2. 0 AA AZ H :P P Z B,C,D   0 BA BZ H :P P Z B,C,D   0 Cj CZ H :P P j A,B;Z B,C,D    0 Dj DD H :P P j A,B,C   Non-rejection Rejection Rejection Rejection H3 tested whether final exam grades reflected continuous assessment grades (Table 5). Consistent with the evidence reported for H2, the H3 results indicate that A-grade students were more likely to achieve A in the final exam than B or a lower grade. However, grade- B students were more likely to drop a grade to C, so we were not able to reject the Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 98 hypothesis that the conditional probability of obtaining the same grade was greater that the conditional probability of obtaining a lower grade. As for grade-D students, the probability of getting a better final exam grade was significantly lower than the conditional probability of maintaining a D grade. Table 5. Test for hypothesis 3. 0 AA AZ H :P P Z B,C,D   0 BB BZ H :P P Z A,C,D   0 CC CZ H :P P Z A,B,D   0 DD DZ H :P P Z A,B,C   Non-rejection Rejection Non-rejection Non-rejection We checked the robustness of our results by examining these across different years. Our evidence for the whole sample held for each year, with the exception of the 2012-2013 academic year where there was a lower probability of gaining an A grade in the final exam conditional on an A grade in the continuous assessment and that the probability of obtaining a B grade conditional on a B grade in the continuous assessment was lower than any other probabilities. We also tested for the possible impact of teacher competence by performing a similar analysis — for the same syllabus, years and assessment procedure — for two other teachers of different groups, obtaining results similar to those reported here. Overall, our results regarding conditional dependence of student final exam grades on continuous assessment grades would indicate that: (a) for the top students, continuous assessment results quite faithfully reflect final exam results; and (b) for more mediocre students, continuous assessment results are indicative of even more mediocre final exam grades. We interpret these results as reflecting how students manage study time: they do not maximize knowledge of the course as a whole but rationally managing their time so as to maximize the probability of a pass while dedicating more time to subjects in which they might perform better. This evidence would cast some doubt on the usefulness of continuous Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 99 assessment as advocated in the Bologna Process as a means to improve learning and educational outcomes. 6. Conclusions Continuous assessment is an important pillar of the Bologna Process aimed at creating a common European Higher Education Area. It is intended to improve student learning and understanding and, ultimately, academic performance. We examined whether continuous assessment results contributed to final exam outcomes by studying conditional dependence between the two kinds of grades for a first-year microeconomics undergraduate course offered at a Spanish public university. Our empirical evidence for courses taught between 2009 and 2015 indicate that continuous assessment grades were a poor indicator of final exam grades, given that the conditional probability of a better grade in the final exam than in the continuous assessment was lower than the corresponding probability of getting the same or lower grade, except for students who attained an excellent continuous assessment grade (A). Final exam grades were generally poorer than would be expected if continuous assessment was fulfilling its goal of enhancing and encouraging student learning. For the microeconomic courses analysed in our study, we conclude that continuous assessment grades did not yield useful information on final outcomes. Our findings may be explained by the time-management choices of students, who may rationally manage their scarce time in such a way that, once they have maximized the probability of a pass (informed by the continuous assessment grade), they dedicate more time to other subjects for which they could probably get a better grade. Our results have implications for the efficiency of resources earmarked for more personalized education modes aimed at improving educational outcomes. Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 100 References Andrietti, A. (2014). Does lecture attendance affect academic performance? Panel data evidence for introductory macroeconomics. International Review of Economics Education 15, 1–16. Bonesrønning, H., Opstad, L. (2012). How much is students’ college performance affected by quantity of study? International Review of Economics Education 11(2): 46–63. Chan, K.C., Shum, C., Wright, D.J. (1997). Class attendance and student performance in principles of finance. Financial Practice and Education 7 (2) 58–65. Chen, J., Lin, T-F- (2015). Effect of Peer Attendance on College Students’ Learning Outcomes in a Microeconomics Course. The Journal of Economic Education 46(4), 350- 359. Christoffersen, P. (1998). Evaluating interval forecast. International Economic Review 39 (4), 841–862. Devadoss, S., Foltz, J. (1996). Evaluation of factors influencing student class attendance and performance. American Journal of Agricultural Economics 78 (3) 499–507. Durden, G.C., Ellis, L.V. (1995). The effects of attendance on student learning in principles of economics. American Economic Review: Papers and Proceedings 85 (2) 343–346. Hill, L. (1991). Effort and Reward in College: A Replication of Some Puzzling Findings, in James W. Neuliep, ed., Replication Research in the Social Sciences. Newbury Park, CA: Sage, 139–56. Krohn, J.B., O’Connor, C.M. (2005). Student effort and performance over the semester. The Journal of Economic Education 36 (1) 3–28. Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2017.6548 Social and Technological Sciences EISSN: 2341-2593 Reboredo (2017) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 4 Nº 1 (2017): 88-101 | 101 Martin, P., Walker, I. (2006). Student achievement and university classes: effects of attendance, size, peers, and teachers. Discussion Paper 2490. IZA. Plant, E.A., Ericsson, A.K., Hill, L., Asberg, K. (2005). Why Study Time Does Not Predict Grade Point Average across College Students: Implications of Deliberate Practice for Academic Performance. Contemporary Educational Psychology 30, p96116. Rau, W., Durand, A. (2000). The Academic Ethic and College Grades: Does Hard Work Help Students to ‘Make the Grade’? Sociology of Education 73: 19–38. Reboredo, J.C. (2014). Can gold hedge and preserve value when the US dollar depreciates? Economic Modelling 39, 168–173. Rodgers, J.R. (2002). Encouraging tutorial attendance at university did not improve performance. Australian Economic Papers 41 (3) 255–266. Romer, D. (1993). Do students go to class? Should they?. Journal of Economic Perspectives 7 (3) 167–174. Schuman, H., Walsh, E., Olson, C., Etheridge, B. (1985). Effort and Reward: The Assumption that College Grades are Affected by the Quantity of Study. Social Forces 63, 945–66. Stinebrickner, R, Stinebrickner, T.R. (2005). How much does studying matter? Federal Reserve Bank of Cleveland Proceedings: 5559.