Microsoft Word - 03-13 Engelbrecht et al FINAL.docx Pythagoras, 72, 3-13 (December 2010) 3 Are OBE‐Trained Learners Ready for University Mathematics?    Johann Engelbrecht, Ansie Harding & Patrick Phiri  University of Pretoria  johann.engelbrecht@up.ac.za, ansie.harding@up.ac.za & patrick.phiri@up.ac.za    The 2009  intake of university students were  the first  to have received complete school  education within the recently implemented Outcomes‐Based Education (OBE) system. A  feature of the matriculation examination results of these students was the exceptionally  high Grade 12 marks for Mathematics. This paper addresses the question of how the 2009‐ intake of students performed at university with respect to general performance, general  attributes,  mathematical  attributes  and  content  related  attributes.  It  appears  that  these  students  are  better  prepared  with  respect  to  personal  attributes  such  as  confidence.  However,  in  many  instances  they  are  weaker  than  their  predecessors  with  respect  to  mathematical and content related attributes.  Yet, there are positive indications that these  students adapt and improve over a semester. We make some suggestions on how to make  the transition from secondary to university mathematics somewhat smoother.  One of the concerns of the Department of Education (DOE) after 1994, was that many learners did not develop the required problem solving skills and the critical reasoning ability during the learning process (DOE, 2000). A new education system became a priority and in 1998 a new education system called Curriculum 2005 was implemented. The key principles of Curriculum 2005 include integration, holistic development, relevance, participation and ownership, accountability and transparency, learner centredness, flexibility, critical and creative reasoning, quality standards and international competitiveness (DOE, 1997). Not only did the OBE system bring about changes in the approach but also a new curriculum. This meant a change at the very heart of the education system. One of the concerns regarding the new education system was to what extent these learners would be prepared for university mathematics. Therefore the university preparedness of the 2009 intake of students received much attention. The first-year intake of the previous three years all had had partial exposure to OBE – these students experienced OBE for a few years but returned to the old curriculum in their final three years of schooling. The 2009 intake was the first group of students who had followed the OBE curriculum for their entire school career. Mathematical university preparedness In a recent study in Ireland investigating students’ inability to cope successfully with the transition between secondary and university mathematics, Hourigan and O’Donoghue (2007) found that essentially there is a big difference between the nature of first-year students’ mathematics experience at pre- university level and that which they experience at university in mathematics intensive courses. They also found that the unpreparedness of students caused permanent damage to students’ further mathematics careers at university. Several researchers have reported on this problem. Craig (2007) and Hoyles, Newman and Noss (2001) reported on a number of studies dealing with the difference between university expectations and the wide spectrum of mathematical abilities of the new students. De La Paz (2005) and Hoyles et al. (2001) name, amongst others, the changing school curricula as one of the reasons for these changes. Are OBE-trained students ready for university mathematics? 4 The focus in this paper is specifically on this facet. Changes in the school curriculum bring challenges for university lecturers and may result in the development and implementation of bridging courses as well as the need for changes to university curricula and assessment strategies (Craig, 2007; Wood, 2001). We have reported on the first stage of our project (Engelbrecht & Harding, 2008), in which we investigated the preparedness of the transition group (the 2008 intake of students, who had had partial exposure to OBE). We found that the transition group was on par in most skills categories and performed even better in geometry, but was lacking in modelling skills. However, the results of the 2008 paper and results presented here are not directly comparable as the instrument used in 2008, namely the Alternative Admissions Research Project (AARP) tests then used for entrance purposes, is no longer used. We have reported preliminary results of the 2009 intake of students in Engelbrecht, Harding and Phiri (2009). This paper expands on this report1 by including additional data and re-interpreting the findings. The 2009 intake of students The 2009 intake of students at university were the first to have written a Grade 12 paper in the OBE system. The matriculation Mathematics examination results were surprisingly positive. More than 47% of those who wrote the examination passed Mathematics compared to about 43% in 2008. Of the learners who passed Mathematics, almost half (63 000) obtained a mark of more than 50% in contrast to the typical 25 000 of previous years whose achievement was similar to that of Mathematics Higher Grade (Keeton, 2009). Among the concerns raised were that the papers were too ‘easy’, and that too many learners achieved distinctions. The Concerned Mathematics Educators (2009) group claimed that the final examination in Mathematics was watered down and has therefore widened the gap between school and university for the top learner. The type of questioning was unchallenging for talented and competent learners and if this standard is going to be used as a benchmark for future examinations it will not adequately prepare young learners to study Mathematics related courses at university level. (para. 2) The Department of Education appointed a panel of experts to evaluate the papers. The Ministerial Panel (2009) reported that those learners who scored 50% or more on the 2008 National Senior Certificate Mathematics papers would historically have passed mathematics on the higher grade in the previous system at 40%. It was also found that there was a lack of differentiation between the 70 – 79% level and the above 80% level and that the paper did not provide enough questions at the “knowledge” level of the taxonomy. Although the Grade 12 results look good on paper there are reasons for concern. The question remains whether the OBE system is sufficiently preparing the learners for university studies. In the National Benchmark Tests Project, which assesses how much of the school Mathematics curriculum has been mastered “a staggeringly low 7 percent of the students [2009 university entrants] who wrote the maths tests were found to be proficient. In other words, they would not need extra help to pass their exams. About 73 percent had ‘intermediate’ skills. The rest, 20 percent, had only the most basic skills and would need long-term, consistent attention” (Smith, 2009, para. 7). The challenge faced by universities is clearly enormous, and the need for curriculum responsiveness is evident. Several lecturers who taught first-year mathematics in 2009 reported on under preparedness of students. For example, Huntley (2009) reported that 71 percent of first-year engineering students had passed in 2008 compared with just 35 percent in the 2009 mid-year examination. In addition, the fact that more Grade 12 learners than ever before had passed in 2008, caused record enrolments in university mathematics courses. This had led to enormous strain on the university’s ability to teach. In short, although learners may have achieved the minimum requirements to allow them to study for a bachelors degree, this does not necessarily mean that they will be able to cope at university (Roodt, 2009). 1 We re-publish here some of the initial data with the permission of the editors of Suid-Afrikaanse Tydskrif vir Natuurskap en Tegnologie and Pythagoras. Johann Engelbrecht, Ansie Harding & Patrick Phiri 5 Mathematics in die OBE system The OBE system resulted in differences in the curriculum and approach. In Mathematics some of the more difficult topics, including geometry, were moved to a third paper, an optional examination paper written by only 11 000 students in 2008 nationwide (Keaton, 2009). Certain topics were excluded from the new curriculum, for instance absolute value and some parts of trigonometry and logarithms. It is easy to determine the differences in content but much more difficult to determine the level of mathematical skills of the learners who passed Grade 12 in 2008. We refer here to mathematical skills such as algebraic manipulation and graphical interpretation. In addition to these mathematical skills there are also a number of personal attributes, such as confidence and work ethics, that may have been either improved or compromised because of the different approach of the OBE system. These attributes could also have an influence on success in mathematics at university. The research question addressed in this paper is: To what extent was the 2009 intake of students prepared for university mathematics with regard to performance, general attributes, mathematical skills, and content related skills? The sample This study is firstly based on a questionnaire completed by a number of experienced lecturers involved with the 2009 intake of mathematics students. It is secondly based on the results of two tests in a first semester calculus course, written by a group of first-year engineering students at a large university in Gauteng, after five and ten weeks of lectures, respectively, hereafter referred to as Semester Test 1 and Semester Test 2. The course had an enrolment of 1282 students. We experienced some administrative problems, but managed to gather and analyse the data of 924 students for Semester Test 1 and 820 students for Semester Test 2, with an overlap of about 750 students. In this calculus course students attended four lectures of one hour each and one tutorial of two hours per week. The entrance requirement in mathematics for these students was level 6 (a mark of at least 70%) in Grade 12 mathematics. We used the questionnaire to determine the experience of the lecturers with the 2009 intake and used these opinions as guidelines for further investigation in analysing Semester Tests 1 and 2. Grade 12 marks distribution In 2007, 22% of students enrolling for this course achieved a distinction (more than 80%) in Grade 12. In 2008 the corresponding figure was 24% and in 2009 it was 55%  dramatically higher than for the previous two years. Note that these percentages include students repeating this course and that the actual percentages for new students were somewhat higher. In fact, in 2009, 72% of the new students enrolling for this course in 2009 achieved a distinction in Mathematics in Grade 12. Figure 1: Percentage frequency distribution of Grade 12 performance 0 5 10 15 20 25 30 35 40 0 10 20 30 40 50 60 70 80 90 100 Performance (%) 2007 2008 2009 Percentage of students Are OBE-trained students ready for university mathematics? 6 Figure 1 shows the percentage frequency distribution of the 2009 first-year-students’ Grade 12 marks. It is clear that the Grade 12 marks of the 2009 intake were exceptionally high relative to the previous two years. Note that all the graphs (Figures 1 – 10) are percentage frequency distributions, with the performance intervals as a percentage (0-100) on the horizontal axis, and the percentage of students in the relevant category on the vertical axis. On the horizontal axis the category 50, for example, refers to performance scores of 41 – 50 and 100 to scores of 91 – 100. Note that the category 0 then refers to scores of exactly 0. General performance in university mathematics In Figure 2 we compare the Grade 12 marks of the 2009 group of students with their performance in Semester Test 1 and Semester Test 2.   Figure 2: Student performance in Grade 12 and in Semester Tests 1 and 2 From Figure 2 it is clear that there was a strong shift to the left from the Grade 12 marks to marks in Semester Test 1. The marks for Semester Test 2 does not display the same strong shift to the left. In fact, the marks distribution for Semester Test 2, as well as for the June examination (see Figure 3) show a slight shift to the right and is more similar to the Grade 12 marks distribution. The Pearson correlation coefficient between Grade 12 performance and performance in Semester Test 1 is 0,37 and for Semester Test 2 it is 0,35, and are both significant (p < 0,001). This implies that Grade 12 can still be considered as a good predictor of university success in mathematics but that students have to realise that their Grade 12 performance will not easily be repeated in university mathematics. The percentage frequency distribution of marks in the June examination is given in Figure 3. From this figure it is clear that students have recovered somewhat. However, we should not conclude too much from this graph, because students with a semester mark of less than 30% did not comply with examination entrance requirements and are excluded, reducing the group size for the June examination to 1144. Figure 3: Student performance in the June examination 0 5 10 15 20 25 30 35 40 0 10 20 30 40 50 60 70 80 90 100 Grade 12 Sem Test 1 Sem Test 2 0 5 10 15 20 0 10 20 30 40 50 60 70 80 90 100 Johann Engelbrecht, Ansie Harding & Patrick Phiri 7 Questionnaire to lecturers A questionnaire was completed by ten experienced first-year lecturers – some involved in first-year mathematics for engineering students, some in teaching the mainstream first-year courses including mathematics, actuarial sciences, financial mathematics and computer sciences and others in the pre- calculus extended programme. Lecturers were asked to express an opinion on aspects in which the 2009 students were better off, were worse off and in which there was no change with respect to the following three facets: general personal attributes, general mathematical attributes and content related skills. General personal attributes Lecturers were almost unanimous in their opinion that the 2009 intake of students had more confidence and were more willing to try. Students had a positive outlook and had confidence in their abilities. It was noticeable that students were not prepared to blindly follow the method suggested by the lecturer, they wanted to “experiment” and do things “their” way. Unfortunately this way of doing goes along with a lack of mathematical rigour and a way of writing that often makes sense only to the student him/herself. Some lecturers felt that students had too much self-confidence and spoke about mathematics too loosely rather than writing it out carefully. When marking tests this personal characteristic of (excessive) confidence was also noticeable. As one lecturer remarked: “Students write down everything that they can think about, without any coherence, and hope that somewhere there will be something that can earn them a mark.” This way of doing was not entirely new for 2009 students but probably more so than in the past. It is also possible that the noticeable confidence of the new students can be ascribed to their high Grade 12 symbols for Mathematics and not only to the influence of OBE. Our findings of increased confidence agree with that of Adler, quoted by Smith (2009, para. 23): “the kids are much better than they were, they are more confident, they are more aware of what they do and don’t know and they are more willing to try.” General mathematical skills There was agreement amongst lecturers regarding the deterioration in general mathematical skills. Lecturers were unanimous regarding decrease in the specific skills of factual knowledge, algebraic manipulation and mathematical formulation. Most lecturers felt strongly that there was a decrease in all these skills while the odd lecturer felt that there was an improvement in graphical manipulation and mathematical intuition. It appears that the self-confidence with which students started out with in the course was not justified and that this was not supported by the necessary mathematical skills. For example, in Semester Test 1 many students made mathematical errors such as e xx xx xx xx eexx 44ln 4lnln)4ln( 3ln ln3lnln 4ln44ln ln44ln3 3     According to lecturers there was not only a noticeable weakening of knowledge regarding the properties of logarithms and logarithm manipulation, but there was also an increase in elementary errors such as 22 42 41)41( 2 1 2 1 2 1     xx xx xx b a ba   Are OBE-trained students ready for university mathematics? 8 Furthermore, there was a particular lack of knowledge and a lack of skills in trigonometry, leading to errors such as 42 1sin  xx There was particular concern regarding the poor ability of students to ‘write’ mathematics. Students wrote little and appeared to be uncertain. Often something that made sense was obscured between nonsensical writing. We distinguished between the following general mathematical attributes: algebraic manipulation, graphical manipulation and interpretation, concept application and interpretation, and basic factual knowledge. We grouped questions in the two semester tests into these categories and investigated students’ performance in each of these components. The performance of students in questions that mainly required algebraic manipulation in Semester Tests 1 and 2 are shown in Figure 4. Figure 4: Performance in algebraic manipulation in Semester Test 1 and 2 In Semester Test 1 the distribution leans to the left. There are too many students that scored less than 50% for these questions. It is expected of students to be fluent in algebraic manipulation to ensure success in university mathematics. For Semester Test 2 there seems to have been a slight improvement in the skill of algebraic manipulation. The distribution leans somewhat to the right and although the marks are still poorer than the Grade 12 marks, there was less reason for concern. Whereas students seemed to be out of their depth in Semester Test 1 as far as algebraic manipulation was concerned, they seemed to have adapted and improved. In Figure 5 we give the performance of students in questions in Semester Tests 1 and 2 that mainly required graphical interpretation. Figure 5: Performance in graphical interpretation in Semester Tests 1 and 2 0 5 10 15 20 25 0 10 20 30 40 50 60 70 80 90 100 Sem Test 1 Sem Test 2 0 5 10 15 20 25 30 0 10 20 30 40 50 60 70 80 90 100 Sem Test 1 Sem Test 2 Johann Engelbrecht, Ansie Harding & Patrick Phiri 9 It is clear that performance improved noticeably from Test 1 to Test 2. Graphical interpretation has always been difficult for students but although it appeared to still be the case in Semester Test 1, the pleasing finding was the remarkable improvement in this category in Semester Test 2. Performance in the other mathematical skills categories for the two semester tests is given in Figure 6. Possibly the best of these categories is the concept application category, especially in Semester Test 2, a surprising finding. It appeared that students’ comprehension was not poor at all but that technical manipulation was the stumbling block. A disturbing observation was that basic factual knowledge was not on par, neither in Semester Test 1 nor in Semester Test 2. Is it perhaps a feature of the OBE generation that they prefer to do but are not so keen to spend the required time on studying? Figure 6: Student performance in factual knowledge and concept application in Semester Tests 1 and 2 The correlation coefficients between student performance in each of the components and their Grade 12 performance are given in Table 1. Table 1: Correlation coefficients between university and Grade 12 performance Semester Test 1 Semester Test 2 General results 0,37 0,35 Algebraic manipulation 0,47 0,28 Graphical manipulation and interpretation 0,39 0,21 Concept application and interpretation 0,34 0,32 Basic factual knowledge 0,27 0,22 These correlation figures are all significant (p < 0,001). The poorer correlation in basic factual knowledge supports the notion that many of the students simply did not study enough, and that this was not only the case for poor students. Content related skills It was not surprising that a deterioration in preparedness was observed regarding topics such as absolute value, trigonometric functions and exponents and logarithms, as it was exactly in these topics that content decreased or was totally omitted in the new curriculum. It appears that students were not only inept with respect to general mathematical skills but also with respect to content related skills. In the empirical investigation we grouped questions that mainly concerned each of the content components and considered student performance in each of these components. Are OBE-trained students ready for university mathematics? 10 The correlation coefficients between student performance in Semester Test 1 in each of the components and their Grade 12 performance are given in Table 2. Table 2: Correlation coefficients between performance in components of Semester Test 1 and Grade 12 General result of Semester Test 1 0,37 Inequalities and absolute value 0,33 Functions and graphs 0,36 Trigonometry 0,38 Exponents and logarithms 0,19 Limits and continuity 0,46 The correlation coefficients between student performance in Semester Test 2 in each of the components and their Grade 12 performance are given in Table 3. Table 3: Correlation coefficients between performance in components of Semester Test 2 and Grade 12 General result of Semester Test 2 0,35 Differentiation  concepts 0,29 Differentiation  technical 0,24 Differentiation  applications 0,29 Limits and asymptotes 0,25 Continuity 0,25 All correlations are significant (p < 0,001) and of approximately the same size, except for the component of exponents and logarithms in Semester Test 1, which caught the lecturers slightly off guard. Exponents and logarithms are in the school curriculum and lecturers assumed that students were equipped with the same level of knowledge as in previous years. This was not the case. Students were initially inept in the properties of logarithms and manipulation with exponents and logarithms, more so than before. The result was that although this topic was included in the school curriculum, the teaching pace at university was such that students floundered with their shallow knowledge. On realising this, effort was made for allocating extra time and to giving more attention to logarithms and exponents. Figure 7: Student performance in inequalities and absolute values, logarithms and exponents and trigonometry in Semester Test 1 Johann Engelbrecht, Ansie Harding & Patrick Phiri 11 Inequalities and absolute value together form a topic to which more time needs to be devoted, as is clear from Figure 7. Too many students performed poorly in this topic. As for trigonometry, although there is a group of students that has mastered the topic, there are too many that scored less than 50%. These students could probably be rescued through a slower presentation pace. The main thread of the mathematics course runs through functions, limits, continuity and differentiation – from the concepts to techniques to applications. We now follow this thread with respect to student performance. For functions, limits and continuity (the start of the topic) the performance in Semester Test 1 is disappointing (see Figures 8 and 9). The level of knowledge in these topics was too low. As a result a decision was taken to spend considerably more time on this topic and to gain the time by moving some of the work to the second semester. Figure 8: Student performance in functions in Semester Test 1   Figure 9: Student performance in limits and continuity in Semester Tests 1 and 2 There was no improvement from Semester Test 1 to Semester Test 2 in the limits topic (see Figure 9). It is clear that the topic is difficult for students. Yet, student performance increased in the continuity category. As was seen in other categories, students initially struggled to find their feet but then improved in performance. The concepts involved in differentiation did not appear to be problematic for the majority of students (see Figure 10), although there was a tail of students for which this finding was not true. This tail seemed to be increasing as the topic was deployed. The performance decreased through technical differentiation to applications. Applications are by nature more difficult but the relatively poor performance in technical differentiation did not bode well for the follow-up topic of integration, which is far more complex. Technical differentiation requires the rigour of applying rules, secured by repeated practice. These students did not seem to be geared for practice, and more practice, until it becomes second nature to do differentiation. 0 5 10 15 20 0 10 20 30 40 50 60 70 80 90 100 Are OBE-trained students ready for university mathematics? 12 Figure 10: Student performance in differentiation in Semester Test 2 Discussion The change to a new education system at school level has been the most significant of the past few decades and it was to be expected that certain difficulties and discrepancies would be experienced. It was important to identify problem areas and address eminent issues. The interface between secondary school and university has to provide for a smooth transition and it would have been fortuitous if the new system decreased the existing gap. This study unfortunately shows that for mathematics, the new system has enlarged rather than decreased the gap. We do not put blame on anyone in this study. The national Department of Education involved higher education continually during the curriculum development process and the problems experienced at the moment are not necessarily someone’s “fault”. We are in a transition phase and need to address unexpected problems as they emerge. The secondary education structures as well as universities should devote attention to addressing the widening gap between secondary school and university mathematics. One cannot accept students for university courses if we know in advance that they are under prepared and will most likely experience difficulties. Possible solutions to smoothen the discontinuity between secondary and university mathematics include o Further research to make a detailed analysis of the problem o Continued close cooperation between university and secondary education authorities o Possible changes in the school curriculum o The possibility of an additional mathematics subject as another of the seven school subjects o Raising of admission requirements at universities o Placing more students in extended programmes at universities o Extension of support programmes at universities o Possible changes in the university mathematics curriculum At university level there is a realisation that the teaching pace should be slowed down and that the curricula in first-year mathematics courses need to be revised. Content in first-year courses has gradually increased with time and this resulted in a quickening of the teaching pace. This trend should be halted and reversed. In some university mathematics courses the emphasis has also shifted to a more theoretical approach for which the new intake of students is clearly not ready. The 2009 intake of students thus forced both universities and schools to pause for stock taking. An outreach should be done from both sides for successful education systems at both levels. Johann Engelbrecht, Ansie Harding & Patrick Phiri 13 References Concerned Mathematics Educators. (2009). Analysis of 2008 Grade 12 Results. Letter to National Department of Education. Retrieved from http://www.mathsexcellence.co.za/letter.php Craig, T. (2007). Promoting understanding in mathematical problem-solving through writing: A Piagetian analysis. (Unpublished doctoral dissertation). University of Cape Town, South Africa. De La Paz, S. (2005). Effects of historical reasoning instruction and writing strategy mastery in culturally and academically diverse classrooms. Journal of Educational Psychology, 97(2), 139-156. doi:10.1037/0022- 0663.97.2.139 Department of Education. (1997). Curriculum 2005. Pretoria: Department of Education. Department of Education. (2000). Instructional leadership. School management teams. Pretoria: Department of Education. Engelbrecht, J., & Harding, A. (2008). The impact of the transition to outcomes-based teaching on university preparedness in mathematics in South Africa, Mathematics Education Research Journal, 20(2), 57-70. Available at http://hdl.handle.net/2263/9941 Engelbrecht, J., Harding, A., & Phiri, P. (2009). Is studente wat in ‘n uitkomsgerigte onderrig-benadering opgelei is, gereed vir universiteitswiskunde? [Are students who have been educated in an outcomes-based approach prepared for university mathematics?] Suid-Afrikaanse Tydskrif vir Natuurskap en Tegnologie, 28(4), 288-302. Available at http://hdl.handle.net/2263/14321 Hourigan, M., & O’Donoghue, J. (2007) Mathematical under-preparedness: The influence of the pre-tertiary mathematics experience on students’ ability to make a successful transition to tertiary level mathematics courses in Ireland. International Journal of Mathematics Education in Science and Technology, 38(4), 461- 476. doi: 10.1080/00207390601129279 Hoyles, C., Newman, K., & Noss, R. (2001). Changing patterns of transition from school to university mathematics. International Journal of Mathematical Education in Science and Technology, 32(6), 829-845. doi: 10.1080/00207390110067635 Huntley, B. (2009). Wits first-year pass-rate for mathematics dropped by 37%! Inflated matric results created unjustified expectations. Retrieved from http://www.mathsexcellence.co.za/papers/Wits_first_year_pass_rate.pdf Keeton, M. (2009). 2008 National Senior Certificate Results – feeling confused? Retrieved from http://www.docstoc.com/docs/19786117/Margie-Keeton---matric-results-2008 Ministerial Panel (2009). Report of the Ministerial panel established to review the Grade 12 NSC Mathematics examination papers of Nov 2008. Retrieved from http://www.mathsexcellence.co.za/papers/Report %20of%20panel%20of%20Maths%20experts.pdf Roodt, M. (2009, September 18). When A stands for abysmal. SAIRR Today. Retrieved from http://sareporter.com/index.php?option=com_content&task=view&id=862&Itemid=1 Smith, J. (2009, August 15). OBE education system produces confident illiterates. Saturday Star. Wood, L. (2001). The secondary-tertiary interface. In D. Holton (Ed.), The teaching and learning of mathematics at university level. An ICMI Study (pp. 87-98). Dordrecht, The Netherlands: Kluwer Academic Publishers.