Conversion of a mathematics course t o CAL: a case study of a large-scale rapid change of resources and organization Maggie Pollock,* Erica McAteer,** Gordon Doughty* and Ian Turner* *Robert Clark Centre for Technological Education, University of Glasgow **Department of Psychology, University of Glasgow During 1994-95, first-year maths for the BTechEd degree at the University of Glasgow was student- centred, teacher-supported A modular online maths course replaced a traditional, lecture-based course. Students worked at their own pace, with timetabled and open access computer classes and/or paper handbooks. The course was evaluated by open-ended measures, and study of examination outcomes, providing us with some pedagogical questions and some recommendations for change. With some adaptation, and with important questions still open, the new course will continue to run. Background Glasgow University's Teaching With Independent Learning Technologies (TILT) project is an institutional initiative funded under Phase 1 of the Higher Education Funding Councils' Teaching and Learning Technology Programme (TLTP), We have developed or adapted and integrated computer-assisted learning programs into a variety of courses in subjects ranging from Accountancy to Zoology. This paper reports a complete change of learning resources and teaching strategies in a Faculty of Engineering course. Overcoming 'not invented here' resistance, we used an externally produced program to deliver all maths teaching over a first-year class, and evaluated effectiveness for both teachers and students. The course is for first-year students in the Faculty of Engineering taking the Bachelor of Technological Education (BTechEd) concurrent initial teacher education degree, taught by the Robert Clark Centre for Technological Education in collaboration with St. Andrew's College, Glasgow. It produces school teachers of Technological Studies, Craft and Design and Graphic Communication. 28 ALT-} Volume 4 Number I Problems The students vary widely in age and past experience, and in maths background. The 1994-95 cohort entered with any of Scottish Higher 'A', ' B ' or ' C \ A-level (pass), HND, HNC, ONC, Scottish Wider Access Programme, or SCOTVEC modules. Most qualifications were recent, but some were years old. For these students, the purpose of their maths learning is to acquire manipulative skills rather than high theoretical understanding. The level aimed for approximates that of first-year engineers. Before 1992, the class was taught separately, with close co-operation between maths teachers and technology teachers. Then partnership changes and timetabling problems put the students in with 200 first-year engineers and spread the (lecture-based) course over two years. The BTechEd students struggled, neglecting other subjects, and 29% (6 out of 21) failed the course and dropped out. Our teaching dilemma, ubiquitous in maths, was how to simultaneously motivate the students by applying maths learning to real life problems of concern to them, and help them to quickly gain the basic skills to do the necessary maths manipulations almost automatically. In practice, students cycle between the two. The duration of a cycle may range from the few minutes required to tackle a real-life maths problem to several months' delay before an application to the maths skills learning emerges in another course. Although the joint class of 200 did cycle between theory, example and practice within the lecture period, our target students were not able to keep up with the manipulative skills. The degree course organizers would have preferred to change to teaching all maths inside the technology courses, as and when needed, but could not get agreement from all course teachers. An efficient and manageable alternative was needed urgently. Solution? We designed a change to learning with long cycle times, providing the first year of a two-year maths course on basic skills, and treating their application in other courses. Integration of the cycles would be provided by two of the authors, who also teach or organize several other courses. We bought in a computer-based maths course: CALMAT, developed by Jean Cook (Cook, 1994, Cook and Hornby, 1995) and colleagues at Glasgow Caledonian University. This provides 50 modules of drill and practice tutorial exercises, with self-assessment tests and a built-in diagnostic examination. A handbook with paper exercises is provided as a supplement (or even alternative) to each module. Although DOS-based (a Windows version is under development), it has a clear and easy-to-use interface, has evolved over many years use in higher-education institutions with students much like our own (Tabor, 1993), and is supported by a maths teaching team. We felt that its pedagogy suited our objectives: to provide practical maths knowledge and skills, applicable in engineering and technology project work and beyond, when students are themselves teaching in secondary-school classrooms. For practical as well as pedagogical reasons, we also needed to provide student-centred learning with independent study resources while being able to access diagnostic information to promote appropriate teacher support. 29 Moggie Pollock et 0/ Rapid conversion of a mathematics course t o CAL Course procedure At the beginning of the course, all students sat a diagnostic test of 20 multiple-choice questions. This allowed individual programmes of work to be scheduled for the first term. A weekly three-hour (non-compulsory) class was timetabled in the computer laboratory for the year's teaching period, which had at least one tutor present. Students were free to use the computer lab at other times during term or vacation, with teaching assistance available - often on demand, but always by appointment Copies of the software could be purchased for home use. No formal lectures or tutorials were run throughout the year, except to go over class- examination outcomes. Two text books were recommended to complement coursework. A repeat of the diagnostic test was taken by students at the end of term 1. During term 2, all students followed the same work programme (rather than individualized programmes), but at their own pace. 'Poor achievers', who reported themselves as falling behind schedule, were given extra tutor support. The course was assessed by a degree examination half-way through the third term. Exemption could be gained by students achieving an average of at least 60% by combining the results of the two class examinations, held at the end of the first term and the beginning of the third term. A summer re-sit examination was also available. Information from students Computer experience and attitudes Maths learning experience and attitudes Confidence about specific maths tasks Course attitudes and experience Skills gain Information from teachers Advantages, disadvantages, questions - questionnaire — focus group — checklist - questionnaire — focus group - diagnostic test and exam marks - interviews - debriefing session Table I: Evaluation targets and methods Evaluation With less than three evaluation staff, working across 20 sites, we could provide no more than a minimal evaluation of the course. Table 1 lists the information sought and the methods used. Students' attitudes to computers, and their knowledge and experience of basic computing, were surveyed. Feelings about learning maths and preferred teaching methods were addressed by informal interviews and a focus-group meeting. Students' statements of confidence about specific mathematical principles and practices, before and after working through relevant modules, were gathered throughout the course. A 'task experience' questionnaire was completed by students during term 1, to get an idea of how the course was progressing from their point of view. At the end of the course, shortly before the examination, a second focus meeting was held. The year's assessment grades were studied. 30 ALT-J Volume 4 Number I Teaching staff, who had collaborated in both the planning and the implementation of the evaluation exercise, as well as in the interpretation of interim findings, were interviewed. Evaluation outcomes Student perspective * Table 2 summarizes findings from questionnaires and interviews. Most of the students came into the course computer-literate, in that 14 had had some training in computer use, either at school or at college, with the software they had used ranging from MacWrite to C++. Most used a computer at least once a week. Modestly they stated themselves as 'novice' more often than 'competent' - almost all said they had at least some confidence about using a computer. Computing experience Attitudes, practical knowledge Confidence with computers Maths Experience Experiences with maths learning Confidence about practical maths Experience of BTechEd course Positive Neutral Negative - high - middling to good - generally negative - increasing with CALMAT - independent learning - own pace - immediate feedback - peer support - learning with computer - three-hour sessions — occasional incorrect feedback, - 'one method' teaching - need for tutorial support when 'losing it' Table 2: Student information summary Students' attitudes to and basic knowledge of computers were surveyed (with permission) by questions from a questionnaire developed by Queen's University, Belfast. Almost all attitudes were positive, and almost all knowledge questions were answered correctly. Their experience with maths learning was varied and, generally, negative. The main impression was that it worried them. Lectures were definitely not favoured: 'Once you get behind, you stay there', 'Two hours and aching fingers', were two students' views about listening and note-taking, a third adding: 'You're concentrating on getting everything down, not on taking it in'. Questioned at the end of term 1, students' experience with CALMAT was positive, with some 31 Maggie Pollock et al Rapid conversion of a mathematics course t o CAL criticisms. The human-computer interaction presented no problems - except where they felt it gave them 'wrong' when it should not (an issue of decimal places), and when they could not retry a question without restarting a complete module. For the majority of students, three hours (with coffee break) at the computer screen was not a problem. However, for a few it was - these students would spend some time at the machine, then work through paper exercises in the relevant handbook: They very much liked the own-pace element, although there were still some fears of 'getting lost' or falling behind. Students were encouraged (rather than required) to complete their recommended exercises to a schedule - by three-quarters of the way through the term, many were behind on this. In general, they rated themselves more confident about maths principles and practice as they went through the course, and most students actively liked the working environment, which they felt included a lot of peer support. Teacher perspective Discussions with teachers during the course and afterwards showed them to be even more positive than the students. Table 3 summarizes the information. The feedback from the evaluation exercises fitted in with their own perceptions. • Importing established and supported software, in this case, has been very cost effective. • N o time needed to prepare and deliver lectures. • Less stressful, allows support structures to be developed where needed rather than globally. • Possible improvement in students' practical application of maths in other courses. • Intend t o provide weekly tutorial sessions and support further 'peer' tutorials at need. • Intend t o closely monitor next year's project w o r k for evidence of practical maths knowledge. Table 3: Teacher information summary Information from diagnostic test and class-examination results The first class-examination posed 40 multiple-choice questions and 10 standard problems. Among the multiple-choice questions were the original 20 questions from the diagnostic test, to measure any improvement. The second class-examination set 10 problem-type questions. The degree examination had the same format as the second class-examination. Table 4 gives an overview of the results. The diagnostic post-test at the end of term 1 indicates some improvement in their skills - the range is wide both before and after, and the central tendency is high. Ten students scored 70% or over at pre-test, so had little scope or need for improvement. Polynomials, equations of straight lines, and trigonometry were all topics which showed a significant overall improvement. 32 ALT-J Volume 4 Number I Diagnostic test - significant overall improvement in scores - from high start 65% (range 30% to 85%) - at end of Term I 76% (range 40% to 100%) - better on some topics than others (e.g. improvement in algebra, co-ordinate geometry and trigonometry) Class exams - December multiple choice 69% - December open questions 36% (some students did not attempt these questions) - May open questions 38% . (five students obtained < 8%) Exemptions I student with > 80% 6 students with > 60% First year degree exam assessment grades ' I student with £ 70% I students with > 60% 3 students with > 50% 4 students with > 45% 5 students with < 35% Resit Exams I student with £ 45% 4 students with > 35% Table 4: Diagnostic test, class- and degree-examination outcomes We are concerned about the outcomes for the 'open' questions - standard mathematical problems which depend on knowledge of different topics covered in the course. We do not yet know whether the material is not teaching, the learners are not learning, or the questions are inappropriate probes of their skills. It should be noted that the average in the May class- examination included five students whose marks were below 8%. Without these marks, the average would have been 48%. The first-year degree examination outcomes were acceptable in that they were on a par with the previous year. It is encouraging that, unlike findings elsewhere (Noss, 1995), entry qualifications were not direct predictors of grades. The top six had entered with a mixture of Highers, HNC and ONC, the lowest six with Highers, HNC, HND and Access. Although the assessment procedures, of necessity, differed from those for previous years, the same external examiner's criteria had been satisfied. It is not, however, possible to make any direct comparisons between the examination outcomes of 1994-95 and the previous year, apart from being encouraged that the number of failures after re-sits fell from six (after normative 33 Moggie Pollock et al Rapid conversion of a mathematics course t o CAL scaling was applied to the raw marks) to four (out of 21) with no normative scaling. A good measure of their learning will be their competence and progress during next year's courses — electronics, energy, design, etc. - where much of their work will depend upon maths 'fluency'. The best predictor of examination performance was the post-test diagnostic scores - those who came out best on this at the end of term 1 got the best degree grades, those at the bottom end stayed there. Conclusions Changing the first-year maths course to student-centred, self-paced learning using a CAL package, plus supplementary written material and textbooks, has been shown to be no worse than if the course were delivered using traditional lectures, and in fact was preferred by both the students and lecturers. The students responded well to self-paced learning and peer self- • help groups developed spontaneously. Staff found it less stressful to supervise the computer labs than to prepare and deliver lectures. They could devote more time to those students who were slow and struggling with the work. Every effort will be made to identify earlier those students who are having difficulty. We intend to improve the learning of these students with the second year of the new course. We are planning fortnightly tutorial classes, led by 'demand issues' and supported by some peer tutoring. We also intend a closer monitoring of students' progress through the modules, though interfering as little as possible with the student-centred, own-pace independent-learning model, which we feel is right for these students. References Cook, J. (1994), 'Bridge the gap with CALMAT', Proceedings of the International Conference on Technology in Collegiate Mathematics, November 1994. Cook, J. and Hornby, J. (1995), 'CALMAT mathematics courseware for access to higher education', Proceedings of the SMC Conference, Stirling, May 1995 (in press). Noss, R. (1995), Reading the Sines, final report of the mid-term evaluation of the Transitional Mathematics/Transmath Project, Institute of Education, University of London. Tabor, J.H. (1993), 'Using CALMAT in "levelling up" teaching', CTI Quarterly Newsletter, 4 (1). 34