Computer-based teaching and evaluation of introductory statistics for health science students: some lessons learned Nuala Colgan, Sally McClean and Bryan Scotney Department of Mathematics, University of Ulster Abstract In recent years, it has become possible to introduce health science students to statistical packages at an increasingly early stage in their undergraduate studies. This has enabled teaching to take place in a computer laboratory, using real data, and encouraging an exploratory and research-oriented approach. This paper briefly describes a hypertext Computer Based Tutorial (CBT) concerned with descriptive statistics and introductory data analysis. The CBT has three primary objectives: the introduction of concepts, the facilitation of revision, and the acquisition of skills for project work. Objective testing is incorporated and used for both self-assessment and formal examination. Evaluation was carried out with a large group of Health Science students, heterogeneous with regard to their IT skills and basic numeracy. The results of the evaluation contain valuable lessons. Rationale Traditionally, the teaching of introductory statistics to Health Sciences students has been based on a series of lectures in conjunction with practical classes using an electronic calcu- lator. The use of a statistical package such as BMDP or GLIM has usually been restricted to advanced classes encountering more sophisticated techniques. However, the increasing availability of user-friendly packages designed for teaching, such as Minitab, has meant that it has become possible to introduce elementary classes to a statistical package at an early stage (Stephenson, 1990). In turn, this provides the opportunity for analysis of real data right from the outset of the student's statistical learning, giving experience of prob- lems as they arise in practice (Singer and Willett, 1990). The use of computers in nursing education is considered to be particularly important as a means of preparing nurses for an increasingly computerized workplace (Yucha and Reigeluth, 1983). A teaching and learning approach which combines the use of statistical software with computer-based tutorials provides flexibility with respect to tailoring information to the specific needs of individual types of users, and allows the incorporation of user- 68 ALT J VOLUME 2 NUMBER 2 interaction (Richards et alt 1991; Barker and Manji, 1991; Bishop et al, 1992). The Computer Based Tutorial (CBT) we have developed at the University of Ulster is concerned with descriptive statistics and introductory data analysis using Minitab, and is part of a suite of CBTs developed under the funding of the Enterprise in Higher Education (EHE) Initiative. The tutorial is targeted specifically at health sciences students, and uses data sets relevant to their subject of study as examples. The tutorials are structured so that they can be used for directed learning and assessment to introduce concepts, and student-centred learning to facilitate revision and the acquisition of skills for project work. Objective testing is also incorporated into the tutorial which may be used for either self- assessment or formal examination. Content The CBT introduces the basic ideas and applications of descriptive statistics, with reference to Minitab in applying statistical techniques. It aims to provide the student with an intuitive understanding of statistical concepts, and a variety of relevant examples are used to illustrate the techniques. The tutorial is designed to develop the student's ability to organize, present and interpret data in a number of ways such as tables, graphs and summaries. In addition, it aims to introduce the student to the concept of statistical inference, and enable him or her to carry out and interpret simple hypothesis tests. Various screens within the tutorial include output from Minitab, with an Explanation button, or a short note which may be expanded to give additional information. An example is presented in Figure 1, illustrating the Minitab command DESC which is used here to produce summary statistics for the pulse rates of respondents who did not run before their pulse was measured. The Explanation button provides information on how to calculate the median and semi-interquartile range. For the non-runners MTB>descC17 produces: N MEAN C17 57 0.105 MIN MAX C17 -14.000 10.000 MEDIAN 0.000 Q1 -2.000 TRMEAN -0.137 03 2.000 jggicirifiii STDEV 4.1 El SEMEAN 0.551 flSJiSWtlOHiij Figure LA typical screen 69 Nuala Colgan, Sally McClean and Bryan Scotney Computer-based teaching On completion of a section, the student may undertake to answer a series of questions for self-assessment. These may require a textual answer, or a choice of one or more appropriate answers from a selection provided. Delivery The tutorial has been developed using version 3.1 of the authoring system Guide running under Windows and under the management of a Novell network which allows the network manager to collect individual student answers from the respective work-stations, and to present them in a file for marking by the class instructor. In our trials, the students were introduced by the lecturer to the system prior to starting to use the CBT, initially in a lecture theatre, and subsequently in a computer laboratory. Each topic was introduced verbally to the students, as a group, before they began to use the appropriate section of the tutorial. In order to facilitate independent learning, the CBT was accompanied by a printed booklet which explained the essential features of the system. The students were allowed to work through the tutorial during class, and in their own time, and both individual and group use of the system was encouraged. The use of the CBT was piloted with a group of 70 nursing students as part of an introductory course on research methods. These students were judged to be particularly appropriate for introduction to the subject via a CBT as they provided a means of allowing a large heterogeneous group with regard to IT skills and basic numeracy an opportunity to familiarize themselves with the material at their own pace. Previous cohorts had expressed dissatisfaction with the course as they found the class size too large and the topic difficult to master, and expressed a lack of conviction that either statistics or computing was needed for nurse training. It was hoped that the CBT could help to overcome some of these problems. Prior to the introduction of the tutorial, students were taught using a combination of formal classes and computer-laboratory sessions. Although it was deemed essential for future project-work that they learned to use the computer for statistical analysis, the laboratory classes often proved problematic as a few students with learning difficulties tended to monopolize the lecturer's time at the expense of the more able students. Under the new system, students used the software as part of a series of one-hour classes held in two inter-connected 30-workstation networked laboratories. During class, the lecturer, assisted by a demonstrator, was present to answer questions and trouble-shoot. Students were allowed to access the software outside formal teaching times. Evaluation At the end of the course, the students were evaluated by means of a confidential questionnaire in order to ascertain how useful they had found computer-based learning, and their views on the system. The questionnaire is presented in Appendix 1. It was completed by the 70 students of whom 67% had managed to complete all four sections of the tutorial, while 17% had completed three out of four during class. 11% had completed two sections, and 5% had completed only one. In all, nearly three quarters of the group managed to complete them during the class, so the volume of material was about right for the amount of time allocated during timetabled contact. Of those students who completed all four sections of the tutorial, 74% often or always completed them during the 70 ALT J VOLUME 2 NUMBER 2 timetabled classes. Overall, 73% never attempted the tutorial outside class. This aspect of students' use of the CBT may be partly accounted for by the fact that 23% of the class were part-time students for whom on-campus time was severely limited, but it is still rather disappointing. In all, 67% of students found the CBT helpful and 33% found them unhelpful. There was no association between success at completing the tutorials and whether a student found them helpful, with 65% of students who failed to complete all four sections finding them helpful, as opposed to 67% of those who completed all four Of2 = 0.038 with 1 degree of freedom). All (27%) students who attempted the tutorial outside the timetabled classes reported that they found it more useful outside than during class. Overall, the response as to how useful the students found the tutorial was again disappointing. 64% said that they did not feel discouraged if they scored poorly on the end-of-section self-assessment tests, and 37% said that they found the tutorial beneficial for reinforcing understanding, 11% for increasing confidence, 54% for providing explanations, 20% for providing a change from Minitab, 39% for reviewing material, 50% for testing understanding. On the other hand, only 3% said that they found the tutorial beneficial for providing understanding. Furthermore, 33% reported that they did not enjoy the tutorial, as opposed to 47% who did not mind it. Only 20% reported that they positively enjoyed it, and only 31% thought they would use computer-based instruction again if given the opportunity, while 44% would not use it. The remaining 25% were unsure. 32% preferred to work alone while 68% preferred to share a computer and discuss the answers with someone else. The multiple-choice questions were found useful by 68%, but only 22% regarded the instructional level to be appropriate to their needs. 55% of students did not find the instructional booklet useful, with no association between this comment and whether they had worked alone or together (x2 = 6.45 with 3 degrees of freedom) or whether they had used the tutorial outside the timetabled classes (g2 = 0.053 with 1 degree of freedom). However, a majority (55%) thought that the tutorials were a useful revision aid, with a further 33% unsure, and only 12% who did not think they were useful for revision. The evaluation questionnaire offered the students the opportunity to raise any additional issues; ones which were frequently mentioned, and which are discussed in the next section, were that the classes were too large, that they were inadequately experienced in using computers, that they preferred ordinary lectures, and that they would like a more comprehensive induction programme prior to using the CBT and Minitab software. Discussion In terms of completing the CBT, the students successfully used the tutorial, with two thirds of them completing all four sections and three quarters of them completing it within timetabled classes, and it is encouraging to note that many students found it helpful, particularly as a means of increasing understanding and as a revision aid. What is more, two thirds of the students found the tutorial helpful, or did not mind it. However, the figure of 44% of students who stated that, if given the chance, they would not use computer-based learning again has provoked us seriously to examine the shortcomings in both the delivery of the tutorial and the way it is embedded in the course. 71 Nuala Colgan, Sally McClean and Bryan Scotney Computer-based teaching Some of the negative comments are concerned with problems of the course rather than the delivery mechanism. For example, students expressed a dissatisfaction with large class sizes, which meant that individual assistance was limited. The most frequently voiced concern, however, was that students felt that much more initial explanation was required from the lecturer of both the structure of the tutorial and its statistical content. Some students expressed a preference for a short verbal introduction at the beginning of each laboratory session; others felt that the whole of the first few classes should follow a traditional lecture/discussion format to establish the teaching and learning of the statistical content, and only then should work be predominantly laboratory based. Despite the fact that the use of Minitab and the CBT requires very little knowledge about the machine or its operating system, many students actually saw the use of computers as a 'complication' with which they felt unable to cope until they had mastered the statistical material by more traditional means. This was certainly at variance with our desire to enable students taught in a computer laboratory to have immediate hands-on experience of the methods being taught, and also with our hope that thus liberated from the drudgery of calculators, they might focus their attention on the statistical methods under consideration. We had anticipated that the teaching method would enable the students, from the outset, to adopt a more exploratory and research-oriented approach with a focus on method- selection rather than calculation. But many students felt that their own limited knowledge of computers was an obstacle to adopting such an approach. While some students felt that by being encouraged to develop a more critical approach to the use of statistics they would be better equipped to cope with project work, they felt that the tutorial would be of greater benefit at a later stage in the course in relation to clinical practice. A small number of students still expressed a lack of understanding as to why they needed to use the computer at all. The clear lessons in all this are that in future years we shall focus more on motivating students at the beginning of the course, and on providinga more structured introduction to the use of the computer and the use of computer-based learning. This may in part be achieved by accepting the need for the students to undertake the learning of basic statistical concepts by more traditional means prior to their exposure to computer-based approaches. Together with a reorganization of delivery which would permit greater access to the CBT outside of timetabled classes, the students would thus 'gain control of their learning' (Isaacs, 1990). In any case, the entire approach must be underpinned by a commitment to the relevance of computer-based statistics to both health science research and clinical practice. References Barker, P. and Manji, K. (1991), 'Designing electronic books', Educational and Training Technology International, 28, 4, 273-80. Bishop, P., Beilby, M. and Bowman, A. (1992), 'Computer-based learning in mathematics and statistics', Computers in Education, 19, 1/2, 131-43. Issacs, G. (1990), 'Course and tutorial CAL lesson design: helping students take control of their learning', Educational and Training Technology International, 27, 1, 85-91. Richards, S., Barker, P., Giller, S., Lemont, C. and Manji, K. (1991), 'Page structures for electronic books', Educational and Training Technology International, 28, 4, 291-301. 72 ALTJ VOLUME 2 NUMBER 2 Singer, J.D. and Willett, J.B. (1990), 'Improving the teaching of applied statistics: putting the data back into data analysis', The American Statistician, 40, 3, 223-30. Stephenson, W.R. (1990), 'A study of student reaction to the use of Minitab in an introductory statistics course', The American Statistician, 44, 3, 231-5. Yucha, C. and Reigeluth, C.M. (1983), 'The use of computers in nursing education, practice and administration', Computers in Education, 7, 4, 223-6. Appendix 1 GUIDE COMPUTER-BASED TUTORIAL It would be very helpful to me if you would provide some feed-back on the GUIDE computer-based tutorial which we have been using during the lab sessions in the Unit NS106. Your views will help me to plan the development and use of computer-based tutorials in the future. I would appreciate you spending just a few minutes answering the questions below. Your answers will be anonymous. PLEASE CIRCLE THE APPROPRIATE ANSWER IN EACH CASE 6. In which of the following ways do you feel that the GUIDE tutorial is beneficial? (circle as many as you want) reinforcing understanding increasing confidence providing explanations providing a change from Minitab reviewing material testing understanding providing interest other (please list) 1. The tutorial contained FOUR topics in the main menu: 1 Summarising Data; 2 Manipulating & Describing Data; 3 Associations for Nominal Data; 4 Scatterplots & Correlations. How many of the four topics have you completed? 1 7. In what ways do you feel that the GUIDE tutorial is not beneficial? (please list overleaf as many as you want) 8. D o you enjoy the GUIDE tutorial? very much quite a bit don't mind not much not at all 2. Have you COMPLETED the GUIDE tutorial topics DURING the timetabled classes always often occasionally never 3 . Have you attempted the GUIDE tutorial OUTSIDE the timetabled classes always often occasionally 4. How helpful do you find the GUIDE tutorial? very fairly not very not at all 5. D o you feel you get more out of the GUIDE tutorial DURING the timetabled classes or OUTSIDE the timetabled classes? during outside 9. If you score poorly on a multiple choice part of the GUIDE tutorial, do you feel discouraged ask for help try again review the material (you may wish to circle more than one) 10. In the multiple choice part of the G U I D E tutorial, how easy do you find it to understand what the questions are asking you to do (as distinct from how easy they are to answer)? very easy easy ok usually occasionally usually difficult difficult 11. D o you prefer to work on the GUIDE tutorial at a computer on your own, or do you prefer to share a computer and discuss the answers with someone else? alone share 73 Nuala Colgan, Sally McClean and Bryan Scotney 1 2 4 0 Are you a student on BSc Nursing BSc Prof. Dev. Nursing Other (please specify) (b) Which year of your course are you in? 1 2 3 (c) Are you Full-time or Part-time? Full-time Part-time Computer-based teaching 13. Please indicate your level of agreement with the following statements about the GUIDE tutorial by circling 5 , 4 , 3 , 2 o r l Strongly Strongly Agree Disagree (a) material is well 5 4 3 2 1 organised (b) information on screen 5 4 3 2 1 is clearly presented (c) instructions on screen are easy to follow (d) multiple choice exercises are useful (e) the instructional level was appropriate to my needs (I) I used the accompanying booklet a lot (g) the accompanying booklet was very useful (h) the GUIDE tutorial is a useful revision aid (1) I would use computer- based instruction again if given the opportunity 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 14. If you have any additional comments, please write them overleaf Thank you for completing the questionnaire. Bryan Scotney (7/5/93) 74