bennett Australian Journal of Educational Technology 1998, 14(2), 75-87. Using interactive multimedia to improve operator training at Queensland Alumina Limited Sue Bennett Central Queensland University Peter Reilly Queensland Alumina Limited Queensland Alumina Limited (QAL) and the Interactive Multimedia Unit (IMU) at Central Queensland University (CQU) have recently completed a joint project involving the design, development and implementation of a multimedia training package. The multimedia tutorial forms an integral part of the training curriculum for alumina production workers at QAL’s Fluid Bed Calciner facility. This paper describes the progress of the project to date and the current and future strategies for evaluation of the training program. An initial evaluation of learner attitudes shows a high degree of acceptance and enthusiasm for the new package. The quality of the multimedia training materials was recently recognised at 1997 Queensland IT&T awards for excellence. Multimedia training Effective training is becoming increasing important to Australian industry for two main reasons - the legal responsibility of a company to ensure that employees have adequate training to carry out their work safely and the need to achieve a competitive advantage through increased productivity (Hosie, 1993). The costs of poor training can be high both in terms of the direct investment in inadequate instruction and materials, and the hidden opportunity costs incurred when employees are not able to work to their full potential (Clark, 1989). Training in many industries has traditionally involved two predominant teaching strategies - on-the-job training by peers and face-to-face instruction (Plant and Shannon, 1993). Recently many companies have become interested in the potential of new technologies to offer more effective and efficient training for their staff. The use of technology is a 76 Australian Journal of Educational Technology, 1998, 14(2) means by which training can be made more flexible and supportive of the principles of adult learning (Hosie, 1993). Interactive multimedia can offer a range of benefits over traditional training approaches by providing improved flexibility, cost- and time- effectiveness, consistency and availability (Frankhauser and Lopaczuk, 1996; Murphy, 1993). Forman (1995) identifies benefits and values in four major areas - organisational benefits, instructional benefits, learning effectiveness and business efficiency. Multimedia can also provide improved and more consistent testing and administration (Sims, 1991). Keppell and Richards (1996) also suggest that self-paced multimedia materials offer a private environment which enables trainees to review the material as many times as they wish. Hosie (1993) identifies the need for courses which are professionally designed and evaluated, adopt effective learning strategies and encourage a self-directed approach. This strategy is also supported by Rossett and Barnett (1996) who maintain that basic instructional design principles must guide the development of computer-based training and recommend an approach based upon analysis, scenarios, discovery and expert advice. Industry context Queensland Alumina Limited (QAL) operates the world’s largest alumina refinery at Gladstone with an annual capacity of approximately 3.4 million tonnes - around 10% of western world production. QAL’s permanent work-force numbers 1100 and the plant operates 24 hours per day, 365 days per year. The Calcination section is supported by 74 employees who comprise four operating teams, two maintenance teams and one management team. The section houses the Fluid Bed Calciner - a complex apparatus used to wash, filter and calcine precipitated hydrate to form alumina powder. The calcining process occurs in a complex arrangement of pipes and vessels through which alumina hydrate, air, gas and fine alumina dust circulate. The operation of the Fluid Bed Calciner is both monitored and controlled by computer. Control room operators analyse trends in the computer readouts and make adjustments to ensure efficient and safe operation. Bennett and Reilly 77 Fluid Bed Calciner operators, known as alumina producers, maintain the operation of the Calciner by responding to requests from the control room operators, carrying out maintenance tasks and undertaking housekeeping duties. Needs Assessment Queensland Alumina operates an extensive training program which covers a range of topic areas with the outcomes of each course linked to the needs of the organisation and the individual. QAL’s aim is to provide relevant training which is delivered just-in-time. Queensland Alumina believed that interactive multimedia could provide an additional tool to enhance the operating skills and knowledge of its Fluid Bed Calciner operators and thereby reduce the loss of production caused by operator error. The specific training issues which could be addressed through the use of multimedia in this case were: • Effectiveness. By using the multimedia tutorial trainees could familiarise themselves with the basic concepts and procedures and make better use of practical training with their mentors. • Instructional needs. The multimedia package could be designed to address the section’s specific learning objectives through the use of animation and self-testing. • Flexibility. The multimedia package could be made available on demand and used by an individual at their own pace. • Consistency. The use of a multimedia package could assist in dispelling inaccuracies by providing an authoritative source of information. QAL saw the advantage of using a specialist multimedia development group and after surveying private and university providers chose the Interactive Multimedia Unit at Central Queensland University. The unit offered proven expertise in developing training materials for large industry-based projects with the added benefit of close proximity - only 1.5 hours by car from the refinery site. The project began with an analysis of the training environment and the needs of the trainees. Fluid Bed Calciner operators must have an 78 Australian Journal of Educational Technology, 1998, 14(2) accurate understanding of the internal workings of the system to correctly diagnose faults and implement troubleshooting techniques. They must react quickly, correctly and safely when responding to requests from the control room operators. An examination of the existing training resources - one print manual, two Powerpoint presentations and a Quality Assurance procedure manual - indicated a need for updating and expanding to improve relevance to the practical setting and provide better support for visualisation of the calcination process. Figure 1: The Fluid Bed Calciner training program. A new training structure was designed which incorporated a multimedia tutorial which could be used before, during and after practical training sessions with an experienced operator. While maintaining the importance of procedural knowledge and technical skills the new training program aimed to develop a better understanding of the underlying operating principles of the Fluid Bed Calciner to achieve a better balance between physical and conceptual skills. Bennett and Reilly 79 To be assessed as competent in the Fluid Bed Calciner program trainees must complete all lessons in the multimedia tutorial and achieve a score of 100% for the assessment tasks. Trainees must also undertake a verbal assessment for the practical component and a written assessment at the end of the tutorial during which they are required to apply their knowledge to realistic situations and identify troubleshooting strategies. Development approach The development of multimedia training materials relies on a collaborative approach which draws upon the skills of people from different fields. This project brought together a group of content and training experts from QAL and instructional and multimedia design professionals from CQU. A project manager appointed by CQU and a project coordinator from QAL provided the main communication link between the two organisations. Queensland Alumina Limited Central Queensland University Project coordinator Subject matter experts Area coordinators Project manager (also the instructional designer) Graphic designer Programmer Audio-visual specialists Photographer Figure 2: The project team. The project followed a development cycle which involved needs assessment, collection of content, design of planning grids (storyboards), editorial review, client review, media design and collection, authoring, beta testing, implementation and evaluation. Rather than being a strictly linear process, the development cycle acts a guide for project managers and stages may overlap or be reiterated as necessary. Each of the lessons were subjected to ‘formative experimentation’ (described by Reeves, 1992a) through expert review and beta testing. This process confirmed the accuracy of the content and the functionality of the interactions. Feedback was collected from the testing group through observation and informal discussion. 80 Australian Journal of Educational Technology, 1998, 14(2) 1. Needs assessment 2. Costing and scheduling 3. Content collection 4. Planning grid development 5. Editorial 6. Client review 7. Revision and sign-off 8. Image collection 9. Digitising 10. Authoring 11. Audio 12. Beta testing 13. Client review 14. Revision and sign-off 15. Delivery 16. Trial and evaluation Figure 3: The IMU development cycle Bennett and Reilly 81 Design and structure The interface used for this project was based on a design model which was developed for the Curragh Multimedia training packages (Plant and Dekkers, 1994) and is based on well established interface design principles (for example see Milheim and Lavix, 1992). These multimedia packages were developed to train operators of heavy mining equipment and were designed to address limited literacy and computer skills through the following strategies: • the use of simple direct language • limiting screen information to a single concept • logical explanations for procedures • use of graphics to support and explain text • use of large buttons • avoidance of icons • limited navigation options (next, back, help, quit, menu and settings). Alessi and Trollip (1991) stress the importance of matching the degree of student control appropriately to the educational level of the student and give general rules for the basic navigation strategies which should be provided for adult learners. These include always allowing control of forward and backward progression, allowing temporary termination and providing support through directions and help. Further development of this model has seen the inclusion of optional, full narration which matches the on-screen text and text/audio help which provides an explanation of the features on each screen. Kenworthy (1993) recommends that information be both visualised and verbalised for poor readers and that supporting audio match on-screen text exactly to allow the identification of unfamiliar words. To further develop the model to suit the needs of the QAL trainees a number of new features were introduced specifically for this project. Clark (1989, p. 120) suggests that ‘processes can be displayed on computer effectively through the use of animation and colour to illustrate the dynamic nature of the process’. The first lesson of the Fluid Bed Calciner training package features a large overview animation which clearly shows all parts of the Calciner apparatus and the flow of hydrate, air etc. throughout. Learners can display or hide the flows, and 82 Australian Journal of Educational Technology, 1998, 14(2) can select a particular section to view a close-up with a further textual/audio explanation. This allows learners to develop an understanding of the process as a whole and the relationship of each stage to the others. It was envisaged that trainees would return to and consult this overview many times as they progress through the more detailed information in subsequent lessons. Each lesson also features safety and environment information to encourage trainees to consider these as integral to their duties. Activities included throughout each lesson are designed to allow learners to assess their own understanding of the material as it is studied by providing immediate feedback. At the end of each lesson a series of assessment questions highlights the most important topics covered and these must be answered correctly to complete the lesson. These tasks concentrate on factual and procedural information that a trainee needs at call. One of the aims of this project was to equip trainees not only with an understanding of how the Fluid Bed Calciner functions but to enable them to apply their knowledge to solve common operational problems. Troubleshooting and problem-solving are heavily emphasised throughout the multimedia tutorial with explanations of common malfunctions, and methods for locating the cause and identifying possible solutions. Data collection A number of site visits to Queensland Alumina were conducted by members of CQU’s multimedia development team to discuss content with subject matters experts, take photographs and shoot video sequences. The course material and media were then collated in a planning matrix which was specifically designed for multimedia development at the Interactive Multimedia Unit (Keppell & Buschgens, 1995). This matrix documents the structure, content and interactions for each lesson and can be reviewed by the client and the subject matter experts. After the iterative process of review was complete the matrix was then used by technical staff to assemble the lessons. This method of storyboarding has proved very successful in increasing efficiency, reducing team member frustration, reducing quantitative error and improving the quality of the final product (Keppell & Buschgens, 1995). Bennett and Reilly 83 Implementation and initial evaluation Wills and McNaught (1996) identify and discuss six broad categories of evaluation for computer-based learning. These include the analysis of quantitative data; assessment of student and staff attitudes and perceptions; formative evaluation of packages; comparison with other teaching methods; evidence of effective learning and; collection of information to make curriculum changes. They conclude that a range of investigation methods both quantitative and qualitative should be used - an approach also proposed by Reeves (1991) and Beattie (1994). The initial evaluation of the package has focussed upon the attitudes of the learners who have used the multimedia tutorial. This information will be complemented by further investigations. To date 26 alumina producers and a number of interested section staff members have completed an anonymous questionnaire after using the tutorial. A range of scaled, yes/no and open-ended questions focus on the degree of acceptance by the trainers and trainees and the perceived suitability of the training materials. Results from the initial survey have been positive with learners indicating that they found the package easy to use, the instructions clear, the tasks and activities valuable for learning, the graphics and images effective and the feedback appropriate. The multimedia tutorial was also compared favourably with other instructional materials on the same subject and all users agreed that they would use a multimedia tutorial in a different area and would recommend multimedia training to others. Only one respondent did not believe that the tutorial improved his understanding of the material. A series of open-ended questions provided an opportunity for further feedback. The following sample comments provide both encouragement and suggestions for improvements: ‘The software is an excellent training/learning tool, but there is no replacement for hands on experience. A picture says a thousand words, this is a step in the right direction for QAL.’ ‘The animations are very helpful.’ ‘I thought that it was well worth the money. I learned a few things I didn't really know.’ 84 Australian Journal of Educational Technology, 1998, 14(2) ‘Quality of photos is rather poor. As offered this package is quite good. Further development is needed. I would like to see another package up and running, but with more detail.’ The high quality of the training materials was recently recognised at the 1997 Queensland Information Technology and Telecommunications awards at which the project received the award for best educational development or application. The award was judged on the basis of the project’s benefits to Queensland, innovation, quality, recognition, local content, potential and currency. Ongoing evaluation A full evaluation of this project requires a long term study which will involve the conduct of interviews, collection of user data and tracking of performance indicators. Interviews will be conducted with trainees selected from each production team to generate an open-ended discussion, particularly about issues raised in the survey process. An interview guide will be used, although some flexibility will be allowed for interviewees to pursue topics about which they feel strongly. It is envisaged that the interviews will encourage learners to draw their own conclusions about the learning experience. Quantitative information will also be collected by the management system which tracks student progress through the package. A log file records information such as the number of times the tutorial is used, the length of time a trainee spends on a lesson and the trainee’s success rate on the activities and assessment. Information will also be collected about the Fluid Bed Calciner operation to ascertain whether the number of unscheduled outages attributable to operator error have decreased since the new training package was introduced. This may provide an indication as to the effectiveness of the materials in increasing operator skill and knowledge. Statistics on damage to the inner lining of the Calciner and consumption rates of materials will also be collected and examined as another possible indicator of training effectiveness. Bennett and Reilly 85 Future directions Since the completion of the Fluid Bed Calciner training package, Queensland Alumina Limited and the Interactive Multimedia Unit have commenced two further training projects. These packages will include indexing features which will allow the tutorial to be used as both a training and reference tool -reflecting a common request among users of the Calciner package. The project managers and coordinator have also received a small grant from Central Queensland University to carry out a detailed case study focussing on project management and communication, content collection, design standards, the development process and evaluation methodologies. This sort of intensive study is recommended by Reeves (1992b) and has been used by Burgess (1995) to describe a similar collaboration between industry and an educational institution. Conclusion The aim of the Fluid Bed Calciner training project is to encourage a better understanding of the underlying processes in the Calciner to allow operators to work more effectively with troubleshooting techniques and reduce the number of unscheduled breakdowns each year. An initial evaluation of the software shows a high degree of acceptance by the target users and has identified some areas for improvement. The continued evaluation of this project over the next year should provide further information about the appropriateness and effectiveness of multimedia training in this environment which will be used to improve the current training package and contribute to on-going research into effective training design. References Alessi, S. & Trollip, S. (1991). Computer-based instruction: Methods and development. Prentice Hall, New Jersey, USA. Beattie, K. (1994). How to avoid inadequate evaluation of software for learning. In K. Beattie, C. McNaught and S. Wills (eds), Interactive multimedia in university education: Designing for change in teaching and learning. Elsevier Science BV, Holland. 86 Australian Journal of Educational Technology, 1998, 14(2) Burgess, J. (1995). An evaluation of an industry training project based on the design, development and use of self-paced multimedia courseware: A case study. MEd thesis, University of Southern Queensland. Clark, R. C. (1989). Developing technical training: A structured approach for the development of classroom and computer-based instructional materials. Addison- Wesley, Reading, Mass., USA. Forman, D. (1995). The use of multimedia technology of training in business and industry. Multimedia Monitor, July, pp. 22-27. Frankhauser, R. and Lopaczuk, H. (1996). Exploring the multimedia landscape from a training and professional development perspective. In J. Hedberg, J. Steele and S. McNamara (eds), Learning Technologies: Prospects and Pathways. AJET Publications, Canberra, Australia. Hosie, P. (1993). Technologically mediated learning: The future of training in Australia. Australian Journal of Educational Technology, 9(1), 69-86. http://www.ascilite.org.au/ajet/ajet9/hosie.html Keppell, M. J. & Richards, J. (1996). Choosing multimedia as a training option: Client Considerations. Paper presented at the Fifth Annual Multimedia in Education & Industry Conference, Charleston, South Carolina, July 22–28. Keppell, M. J. & Buschgens, R. A. (1995). Optimising communication between instructional designers, graphic artists and computer programmers in the development of multimedia materials. In A. C. Lynn Zelmer (ed), Higher Education: Blending tradition and technology, Proceedings of the 1995 Annual Conferences of HERDSA. Central Queensland University, Rockhampton, Australia, pp. 439-451. Kenworthy, N. (1993). When Johnny can’t read: Multimedia design strategies to accommodate poor readers. Journal of Instruction Delivery Systems, Winter, 27- 30. Milheim, W. D. and Lavix, C. (1992). Screen design for computer-based training and interactive video: Practical suggestions and overall guidelines. Performance and Instruction, 31(5), 13-21. Murphy, L. (1993). DVI in Organizational Information Retrieval and Training. In D. M. Gayeski (ed), Multimedia for Learning - Development, Application, Evaluation. Educational Technology Publications, New Jersey, USA. Plant, H. & Dekkers, J. (1994). The use of multimedia to train mine operators at Curragh Queensland Mine, Blackwater Central Queensland. Central Queensland Journal of Regional Development, 3(2), 53-61. Plant, H. & Shannon, B. (1993). The use of multimedia to train mine operators at Curragh Queensland Mine, Blackwater Central Queensland. Internal report to the Division of Distance and Continuing Education, Central Queensland University. Bennett and Reilly 87 Reeves, T. (1991). Ten commandments for the evaluation of interactive multimedia in higher education. Journal of Computing in Higher Education, 2(2), 84-113. Reeves, T. (1992a). Evaluating interactive multimedia. Educational Technology, 52(5), 47-53. Reeves, T. (1992b). Research foundations for interactive multimedia. In Proceedings of the International Interactive Multimedia Symposium, Perth, Western Australia, January 27-31, 1992, Promaco Conventions Pty Ltd, Applecross, Western Australia, pp. 177-190. http://www.ascilite.org.au/aset-archives/confs/iims/1992/reeves.html Rossett, A. & Barnett, J. (1996). Designing under the influence. Training, 33(12), 34-42. Sims, R. (1991). Computer based training: a handbook for professionals. Knowledgecraft Pty Ltd, West Chatswood, Australia. Wills, S. and McNaught, C. (1996). Evaluation of computer-based learning in higher education. Journal of Computing in Higher Education, 7(2), 106-128. Sue Bennett is an instructional designer in the Interactive Multimedia Unit at Central Queensland University’s Rockhampton campus. Sue’s email address is s.bennett@cqu.edu.au. Peter Reilly is the People and System’s Coordinator of Queensland Alumina Limited located in Gladstone. Peter’s email address is reillyp@qal.com.au