parr
Australian Journal of
Educational Technology
How successful is "Successmaker"? Issues arising
from an evaluation of computer assisted learning in a
secondary school
Judy M. Parr
University of Auckland
The paper discusses issues arising from the year long evaluation of the use
of the computer assisted learning (CAL) package "Successmaker" in a
secondary school (Years 8-12). All students in Year 8 (n=199) and Year 9
(n=180) had equal access to the computer assisted learning lab for an hour
session a week. Specialised groups like learning support, special education
and ESL also used the lab for blocks of time. Multiple sources of data were
used to assess the process and outcomes of introducing CAL. Issues arising
include whether the rationale for such a program is well founded; its
effectiveness, including cost effectiveness and generalisability of learning,
and the question of educational integrity in terms of curriculum and
pedagogical appropriateness.
The rationale for introducing an integrated computer assisted learning
(CAL) package like "Successmaker" (1) into a school is likely to be multi-
dimensional. For the principal of the secondary school where the
evaluation of "Successmaker" took place (2), the main reason was to help
address the issue of under-achievement in literacy and numeracy by a
significant proportion of students. The main mechanisms by which
successful outcomes might be achieved were considered to be the interest
value of the medium, whereby students may be more prepared to tackle
work; individualisation allowing instruction and practice at an
appropriate level, and the experience of success which would result from
this. Computer assisted learning, as Apple (1991) comments, is often seen
as an approach to the problems of under achievement and relevance,
Parr 21
although Apple questions whether it is part of the solution or part of the
problem. This paper will examine the use of "Successmaker" as a solution
and raise some of the problematic issues.
Cognitive learning research (eg. Beck, Perfetti & McKeown, 1982;
Schneider & Schiffrin, 1977) suggests that, in order to perform complex
intellectual tasks such as reading and mathematical problem solving,
many of the subskills have to be automated. Practice is the way to achieve
this. The concept of practice is entrenched in pedagogy; basic facts and
table drills in maths and sending books home to read are examples from
the early school years. Computer assisted learning delivers practice
examples most efficiently; the computer is infinitely patient in the
presentation of example after example until mastery is attained. Although
practice can be delivered efficiently via CAL, the question of effectiveness
remains.
Most of the sample of 63, Year 8 students interviewed during the
evaluation of "Successmaker" were positive about CAL. A number (26)
commented on the interesting or different or fun nature of the work. For
ten of these students it was enjoyable simply because they did not have to
write - "It's better just to push the buttons. Here you have more time and
relax". Another commented "It's more interesting than you do in class;
there's graphics and things like that". However, for some, interest was
detrimentally affected by features of the program like repetition and pace.
"Boring" was a favoured descriptor, offered by nearly a third of the
interviewees. When probed, it seemed to be either that the student readily
grasped the concept but had to do a minimum number of examples before
progressing ("I hate it when you really know the stuff but it keeps coming
up") or that, despite not having mastered an idea, the student found the
presentation of more of the same tedious ("The questions repeat over and
over" and "They have a lot of one sort of question, not a variety").
Although the material presented in the strands (3) based programs is
individualised and varies according to ability, a few teachers felt that for
some able students the progression was too slow and the work
insufficiently challenging. Student comment supported this: "You have to
work through all the programs. It's easy but it takes too long and it gets
boring". It is possible for teachers to adjust levels manually but this
requires that the teacher learn to recognise the need for intervention in a
novel situation and to use different sources of diagnostic information.
Besides ability differences among students, there are likely to be
22 Australian Journal of Educational Technology, 1995, 11(1)
significant individual differences in saturation level in terms of tolerance
for continued examples of a similar kind. In the classroom, the skilled
teacher avoids tedium by finding a new format to practise the same skill or
by changing focus and coming back to the difficult area later. A
"Successmaker" course like "Maths concepts and skills", hierarchically
organised on mastery principles, is less flexible.
Practice is most effective in meaningful, relevant contexts. A criticism of
"Successmaker" and similar CAL programs is that the practice does not
take place in such contexts. The fact that technology, in the form of a
computer, is used to deliver the content does not render the work
automatically meaningful, despite parental beliefs that their offspring
were learning "relevant things" about computers through using
"Successmaker". Nor, as is often naively assumed, does the use of the
computer per se, with its associations with the work place, with
amusement and "hi-tec", encourage students to have more favourable
attitudes to learning. In a study which examined the effects of personal
laptop computer use on attitudes and achievement, Morrison, Gardner,
Reilly and McNally (1993) found that the positive impact of high access to
computers on attitude to school and to a subject area was marginal.
Positive associations occurred where the process based work on the
computer was seen to transfer directly to the content domain of the
discipline. The implication is that the work done on the CAL program has
to bear direct relevance to the associated curriculum area. Teachers in the
evaluation found most satisfactory those sections within the maths
program like "number basics" which paralleled the syllabus for Year 8.
Students, too, based many of their favourable comments about their
progress on the CAL maths unit concerning measurement, which matched
the curriculum closely.
In the evaluation school, "Successmaker" was largely utilised within an
existing, successful learning support curriculum framework. The
educational integrity of any innovation is dependent on the extent to
which it complements and extends curricula goals and melds with the
school's philosophy of teaching and learning and accepted pedagogical
models. Some of the teachers interviewed felt that the dominant
pedagogical approach of "Successmaker" (risking over simplification as
there are a variety of different courses, it could be described as
Parr 23
individualised instruction based on mastery learning principles) was not
sound; others commented that it did not blend readily with their
classroom practice, which often included cooperative work, while others
acknowledged that the approach of the software was different to theirs but
it made for variety. Research suggests that different preferences with
respect to learning styles, like the finding that girls prefer a social
situation, may mean that students benefit differentially from computer
assisted learning (Hattie & Fitzgerald, 1987). However, the data from the
evaluation show that girls made more rapid progress than boys and there
was no difference ethnic groups.
One pedagogical feature which some staff and students commented
positively on was the fact that computer assisted learning allowed
students to take responsibility for their own learning. Researchers (eg.
Clarke, 1990) have commented on the fact that one of the appealing
features of CAL is its opportunity for control. Students, in the interviews,
in outlining what they liked about CAL, commented on a perceived lack of
teacher monitoring and control. This reaction was particularly marked
among the learning support students (low achievers) and may reflect some
of their experiences of education to date. In reality, the students controlled
neither content nor pace to any extent. But, the pattern in the lab was
established; they did not need teacher direction to begin or to log in to the
system. They had some, limited choice about what area to begin work on.
They recorded their own progress scores as they completed a unit.
Teachers commented that students felt that the feedback from the program
was objective and they were more ready to acknowledge their weaknesses.
Thus, the teacher's role may change with the use of computer assisted
learning. Ideally, computer assisted learning with provision for
individualised instruction and practice should allow the teacher more
flexibility to help, particularly those less able students. Schofield and
Verban (1988) found in a CAL situation that teachers shifted away from
concentrating on better students to the point where the weaker students
received four to five times as much attention. But teachers need time and,
possibly, guidance to adapt their role to this new learning situation. In
some cases (Asker, Yavuz & Koksal, 1992), the reported help from teachers
was less during CAL than in the traditional classroom. Classroom
24 Australian Journal of Educational Technology, 1995, 11(1)
observation during the evaluation and the fact that few teachers utilised
the diagnostic features of "Successmaker" supports the view that teachers
have to adjust to a new role which includes, amongst other things,
developing a new set of criteria for assessing on-task behaviour and new
strategies for monitoring the process of practice and learning outcomes.
With a large suite of programs such as "Successmaker", written from a
United States curriculum perspective, many teacher hours also need to be
devoted to exploring the offerings in order to best utilise the material
effectively. As a maths teacher noted, "The material on "Successmaker" is
organised in a different way (to the Year 8 syllabus) in that on
"Successmaker" you get several things at once while our curriculum is in
blocks". Some teachers saw the possibility of the software for "ongoing,
systematic maintenance". This pattern of utilisation requires careful
planning and monitoring in terms of timing to complement the curriculum
yet satisfy individual needs and duration to find the optimum exposure
time. Both the need for curriculum match and the necessity for training
and guidance, alluded to above, certainly need to be borne in mind when
assessing cost effectiveness, as does the observation, discussed below, that
those who made the greatest absolute gains not only had most time on the
system (in addition to normal class time in the subject area) but a most
favourable student teacher ratio, half that of normal classes.
The foremost consideration, in respect to effectiveness, is does the
instruction and practice delivered via CAL lead to enhanced learning
outcomes? Students experienced success within "Successmaker". The most
successful were those who had intensive time and were within small
learning support classes, staffed by dedicated and expert teachers.
Working at their own level certainly allowed students to achieve in terms
of the objectives and criteria of the particular program within
"Successmaker". If the results of Year 8 and Year 9 classes are averaged,
students gained, for every hour spent working on CAL, 1.25 months of
school learning in maths and 3.1 in reading. This is in excess of the claims
made for the software (that of one month of school learning per hour on
the program), although it should be remembered that these gains are
occurring while the normal school program is in operation and over a time
period and so are confounded by normal developmental gains.
Those students who made the greatest absolute progress (although at a
slower rate) were the learning support students, students in the lowest
Parr 25
percentiles for literacy and numeracy on nationally normed tests. These
students received intensive work on CAL (four hours a week over a term
and often a further term), working closely with their learning support
maths or English teacher in classes of 12 to 15. The outcome was basically
that they progressed at a rate more approaching average rather than the
slow rate of progress they had exhibited over previous schooling. On
going, longitudinal research would be necessary to establish whether these
students would catch up with their cohort or simply not fall further
behind and whether they would need to continue with intensive work to
achieve this.
Partly as a result of experiencing success, but perhaps also because they
felt they had some control over the process, many students developed
more positive perceptions of themselves as learners. They were achieving
at their own level and felt pleased with their progress "on the computer".
Some of the comments, at interview, suggested there were other spinoffs
in terms of a more confident approach to learning in the normal classroom.
Comments such as "Now in class I can understand things better" and "It's
helped me when I have normal maths" suggest a transfer of learning. And
several teachers were able to nominate specific skills students had
acquired and transferred. However, the perception of generalisation of
learning from CAL to classroom was not borne out by the achievement
data. Results with respect to generalisation of learning in maths, for
example, to school based tests were statistically insignificant although the
trend was in the expected direction.
There are other issues to consider when a school is looking at a costly
piece of software like "Successmaker". These include questions of equity.
Does a school aim for equality of access or is the consideration equity in
terms of outcomes, a goal which suggests that those most in need receive
the most? The evaluation school tried to give all Year 8 and 9 access by
apportioning time equally to all classes. But, as the primary aim was to
help those in Year 8 needing most assistance in literacy and numeracy,
these students received intensive time. This decision had to be made
within the constraints that the license cost of the software was linked to
the number of machines.
A consideration of the balance sheet in order to assess the success of
"Successmaker" shows it, like the rationale for the introduction of CAL, to
be multi-dimensional. Students undoubtedly make progress on the
program, yet there is no strong evidence of generalisation. Those
26 Australian Journal of Educational Technology, 1995, 11(1)
underachieving in literacy and numeracy begin to achieve at a rate
approaching the average but the resource input, both in terms of CAL time
and teacher time, is great. In any consideration of cost effectiveness this,
plus teacher development and teacher time to effect integration, need to be
weighed against other viable alternatives. The majority of students like
CAL; there is evidence of positive affective outcomes. Administering the
correct "dose" is, however, a delicate and vitally important aspect.
Pedagogical features of the program are questioned by some staff and
students and there is a clear need to investigate this and the question of
curriculum match further.
Notes
1. "Successmaker" is the name under which a suite of computer assisted
learning programs, covering the curriculum areas of maths and
science,reading, language skills, basic competency, and computer
education, are marketed. The programs are produced by the Computer
Curriculum Corporation, a division of Simon and Schuster, USA.
2. A detailed report of the evaluation is contained in Parr (1994).
3. Courses are either strands where the student works through
hierarchically organised sets of content/skill areas or lessons which
may be selected to cover specific skills or content. For example, "Maths
concepts and skills " is a course with 16 different strands, including
addition, decimals, fractions etc. An example of a lesson is "The use of
the apostrophe".
References
Apple, M. (1991). The new technology: Is it part of the solution or part of
the problem in education? Computers in the Schools, 8(1/2/3), 59-81.
Askar, P., Yavuz, H. & Koksal, M. (1992). Students' perceptions of
computer assisted instruction environment and their attitudes towards
computer assisted learning. Educational Research, 34(2), 133-139.
Beck, I., Perfetti, C. & McKeown, M. (1982). The effects of long-term
vocabulary instruction on lexical access and reading comprehension.
Journal of Educational Psychology, 74, 506-521.
Parr 27
Clarke, V. (1990). Sex differences in computing participation: Concerns,
extent, reasons and strategies. Australian Journal of Education, 34(1), 52-
66.
Hattie, J. & Fitzgerald, D. (1987). Sex differences in attitudes, achievement
and use of computers. Australian Journal of Education, 31(10), 3-26.
Morrison, H., Gardner, J., Reilly, C. & McNally, H. (1993). The impact of
portable computers on pupils' attitudes to study. Journal of Computer
Assisted Learning, 9,130-141.
Parr, J. (1994). Evaluation of the computer assisted learning project at Rutherford
High School. Wellington, New Zealand: Ministry of Education.
Schneider, W. & Schiffrin, R. (1977). Controlled and automatic human
information processing: II. Perceptual learning, automatic attending
and a general theory. Psychological Review, 84, 127-190.
Schofield,J. &Verban, D. (1988). Computer use in mathematics teaching:
Issues which need answers. In D. Grouws & T. Cooney (eds), Effective
mathematics teaching, Vol 1, pp 169-193. Hillsdale, NJ: Erlbaum.
Contributor: Dr Judy Parr currently teaches and researches in the
Education Department at the University of Auckland. A cognitive/
developmental psychologist, her present research interests include issues in
the development of written language and in the use of technology in the
teaching and learning process. Her address is Education, University of
Auckland, Private Bag 92019, Auckland, New Zealand. Email
jm.parr@auckland.ac.nz
Please cite as: Parr, J. M. (1995). How successful is "Successmaker"? Issues
arising from an evaluation of computer assisted learning in a secondary
school. Australian Journal of Educational Technology, 11(1), 20-27.
http://www.ascilite.org.au/ajet/ajet11/parr.html