How to cite item: Widjanarko, D., Sofyan, H., & Surjono, H. (2016). Improving students’ mastery on automotive electrical system using automotive electrical multimedia. Research and Evaluation in Education, 2(1), 71-78. doi:http://dx.doi.org/10.21831/reid.v2i1.8219 Research and Evaluation in Education e-ISSN: 2460-6995 Research and Evaluation in Education Volume 2, Number 1, June 2016 (pages 71-78) Available online at: http://journal.uny.ac.id/index.php/reid IMPROVING STUDENTS’ MASTERY ON AUTOMOTIVE ELECTRICAL SYSTEM USING AUTOMOTIVE ELECTRICAL MULTIMEDIA ¹Dwi Widjanarko; ²Herminarto Sofyan; ³Herman Dwi Surjono ¹Semarang State University; 2,3 Yogyakarta State University ¹dwi2_otosmg@yahoo.com; ²hermin@uny.ac.id; ³hermansurjono@uny.ac.id Abstract This research was conducted to study the improvement of students’ understanding of automotive electrical system by applying automotive electrical multimedia. The multimedia was developed and validated by automotive and multimedia experts before being applied in teaching and learning processes. The research design was a single group pretest-posttest design with three field testing. The results showed that in the preliminary field testing, the students’ understanding increased by 32.55%; in the main field testing, the understanding was up to 64.89%; and in the operational field testing, the understanding became 77.36%. This indicates that automotive multi- media could increase the students’ understanding of automotive electrical system significantly. Keywords: automotive electrical system, multimedia mailto:dwi2_otosmg@yahoo.com mailto:hermin@uny.ac.id mailto:hermansurjono@uny.ac.id Research and Evaluation in Education 72 − Volume 2, Number 1, June 2016 Introduction The advanced development of auto- motive systems mainly occurs in electrical systems. Most of the systems in automotive field have been controlled electrically. That is why the students of automotive programming study program have to master automotive electrical systems. Unfortunately, based on the survey carried out on automotive teacher stu- dents of Semarang State University (SSU) and Yogyakarta State University (YSU), auto- motive electrical system is one of the subjects which are difficult to study. The main dif- ficulty of mastering automotive electrical systems is to understand automotive electrical circuit and how the circuit works. A research conducted by Widjanarko and Abdurrahman (2006) shows that stu- dents’ ability in explaining and analyzing the operation of automotive electrical systems was disappointed for the mastery level was lower than 50%. Ideally, it minimally reaches 70%. Based on that problem, a big solution is urgently needed to improve the students’ mastery of automotive electrical systems. An instructional media or multimedia has to be developed to facilitate the automotive elec- trical system learning. Multimedia is the integration of more than one medium into some forms of com- munication or experience whcich is delivered by a computer. Most often, multimedia refers to the integration of media such as text, sound, graphics, animation, video, image, and spatial modeling into a computer system (Reeves, 1998, p.22). Multimedia is the use of several media to present information. The combination may include texts, graphics, animation, pictures, video, and sound (Ivers & Barron, 2002, p.2). Computer-assisted edu- cation has been used frequently in modern educational systems because of its benefits like providing persistence in learning in general, providing a learner-centered learning process, getting the event of learning out of four walls and making it independent from space and time, providing the possibility to practice frequently and providing a quick access to information (Kayri, Gencoglu, & Kayri, 2012, p.59). The use of the computer as a medium for instruction provides many capabilities that cannot be readily duplicated within the trad- itional lecture format. Teachers could com- bine the Internet, projector, presentation soft- ware, and web resources into the teaching- learning scenarios. The contents of teaching become vivid with delightful visual icons, graphs, or appealing explanations, which may generate very different teaching situation in contrast to the traditional teaching (Lu & Cheng, 2012, p.1013). Multimedia can be defined as an integration of multiple media elements into one synergetic and symbiotic whole that results in more benefits for the end user than any one of the media element can provide individually (Reddi & Mishra, 2003, p.4) and enable learners study anytime and anywhere (Arkorful & Abaidoo, 2014, p.403) ‘Multimedia’ and ‘interactive multi- media’ are defined by four basic character- istics (Garrand, 2006, pp.4-5), namely: (1) combination of many media (video, text, audio, and still pictures into a single piece of work); (2) computer mediated, or computer which is used to mediate or make possible the interaction between the users and the material or media being manipulated; (3) media- altering interactivity, or user interactivity in multimedia which is best defined as ‘the ability of the user to alter media he or she comes in contact with’; and (4) linking, or allowing links or connections to be made between different media elements. Educational multimedia features used for instructional purposes cover (1) screen design (visual elements: color, text, graphics, and animation); (2) learner control and navigation; (3) use of feedback; (4) student interactivity; and (5) video and audio elements (Stemler, 1997, p.339). There are really only five basic types of media objects we will generally use (Simkins, Cole, Tavalin & Means, 2002, pp.13-14), among other things are (1) images, which come in many forms including graphs, maps, photographs, and drawings; (2) text, which includes everything from image captions to paragraphs of infor- mation; (3) sound, voice recordings, music, and sound effects that can be used alone or to Research and Evaluation in Education Improving students' mastery on automotive electrical system... - 73 Dwi Widjanarko, Herminarto Sofyan, & Herman Dwi Surjono enhance another media element; (4) motion, which includes cartoon-type animation, video, and moving transitions between screens of media; and (5) interactivity, which means making buttons, hyperlinks, and the like. To have a good multimedia, some systematic steps must be carried out. Frey and Sutton (2010, p.491) explain several steps to develop multimedia: (1) define the instruct- ional goals, objectives, and audience; (2) review and investigate the existing options; (3) determine format, budget, and timeline; (4) determine the content, activities, and assess- ment strategies; (5) develop evaluation strat- egies, criteria, and instruments to determine the effectiveness of the project; (6) develop the flowchart, site map, and/or storyboard; (7) develop a prototype; (8) perform a form- ative evaluation; (9) complete the design; and (10) perform a summative evaluation of product and process. A technological approach which in- cludes multimedia application has become in- creasingly important for students in schools. The use of the multimedia opportunities makes the instruction readily available, more affordable, and limitless accessible, and easily comprehensible. The role of technology in every step of the instructional systems is the requirement of today’s world (Eristi, 2008, p.832). Multimedia can be used in instruction by utilizing computer in teaching and learning activities. This learning is termed computer- based learning (CBL). Nazimuddin (2014, p.185) states that computer in education is a diverse and rapidly expanding spectrum of computer technologies that assist the instruct- ional process. The application of computer in instruction includes guided drill and practice exercises, computer visualization of complex objects, and computer-facilitated communi- cation between the students and teachers. Basic features in which CBL environ- ments outwit other forms of learning environ- ments can be summarized as follows (Sidhu, 2010, p.53): (1) the speed in which the computer can respond to individual learner’s need; (2) the way that the computer can offer, and respond to, a wide range of learner inter- action; (3) the potential to represent infor- mation in a wide scope of formats from text to video; and (4) the opportunity to provide unlimited choice of learning paths. Concern- ing the difficulty of mastering automotive electrical systems, this research was conducted to developed a multimedia program able to help students in learning all automotive electrical materials. This multimedia is intend- ed to make automotive electrical systems easy to learn. This article will discuss the improve- ment of students’ mastery on automotive electrical systems during preliminary field test- ing, main field testing, and operational field testing. Method The automotive electrical multimedia was developed following the 10 steps of de- veloping educational product (Borg & Gall, 1989): (1) research and information collecting, (2) planning, (3) developing preliminary form of product, (4) preliminary field testing, (5) main product revision, (6) main field testing, (7) operational product revision, (8) oper- ational field testing, (9) final product revision, and (10) dissemination and implementation. The multimedia was developed based on the need assessment from research and information collecting, planning, and develop- ing preliminary form of product. Before the product was field tested, it was validated by automotive electrical and multimedia experts. The research was conducted from March 20th to October 25th, 2012 at the Automotive Engineering Education Study Program, Yogyakarta State University and Semarang State University – Indonesia. The research subjects in the preliminary and main field testing were 80 students who had passed the automotive electrical subject. In the pre- liminary field testing, six randomly determin- ed students were involved. In the main field of testing, 42 randomly determined students participated. For the operational field testing, 100 students who are programming auto- motive electrical subject were involved. The one-group pretest-posttest design was used in the experiment. A multiple choice test was utilized to measure the automotive electrical system mastery before and after the application of the multimedia. The t-test was used to analyze Research and Evaluation in Education 74 − Volume 2, Number 1, June 2016 the difference of the average scores. The descriptive statistic (%) was also used to describe the level of improvements. Findings and Discussion The automotive electrical multimedia developed in this research can facilitate all automotive electrical systems needed by the students of programming automotive elec- trical subject. The multimedia contains basic electricity, basic electronics, engine electrical system (covering starting system, charging system, and ignition system), and body elec- trical system (lighting system, wiper system, power window system, and power mirror system). Each system presented covers funda- mentals, components and their fuctions, cir- cuits, operation of each component, system operation in animation forms, exercises, and evaluations. The multimedia CD contains 31 folders, 128 power point files with a total of 2007 slides, 31 PDF files, 348 GIF animation files, 1120 WAV sound files, and CD capacity of 618 MB. The developed automotive electrical multimedia must fulfill valid criteria either from substantial view or multimedia point of view, so the multimedia has to be validated. The validation of the multimedia was carried out by four automotive electrical experts was intended to make sure that the content is valid, correct, and appropriate with the auto- motive electrical competencies, so the multi- media can be utilized in automotive electrical subjects. The content validation of the multi- media referred to the substantial validation instrument that had been prepared in this research. The summary of automotive elec- trical expert validation is listed in Table 1. The validity determination of the auto- motive electrical multimedia refers to the value classification to be level having evalu- ative meaning (i.e. good-bad, high-low, valid- invalid, etc.) according to Azwar (2003, p.157). Based on the classifying calculation, criteria were found as follows: the multimedia is invalid if Mpb ≤ 3.79; less valid if 3.79 < Mpb ≤ 4.23; valid if 4.23 < Mpb ≤ 4.68; and very valid if 4.68 < Mpb ≤ 5.12. The data in Table 1 indicate that the multimedia has Mpb of 4.45 and it lies at valid criteria. Therefore, the multimedia is substantially valid. The validation of multimedia based on its performance was also done. The validation was carried out by two multimedia experts to ensure that all aspects that have to be in a multimedia have been fulfilled, so the multi- media could be used in the teaching and learning processes of automotive electrical subjects. The summary of multimedia expert validation is tabulated in Table 2. Based on the classified calculation, the following criteria were found: the multimedia is invalid if Mpb ≤ 3.53; less valid if 3.53 < Mpb ≤ 4.05; valid if 4.05 < Mpb ≤ 4.58; and very valid if 4.58 < Mpb ≤ 5.10. The data from Table 2 indicate that the multimedia has Mpb of 4.45 and it lies at valid criteria. There- fore, the performance of the multimedia is valid. The automotive electrical multimedia was applied in teaching and learning processes to improve students’ competency. The data of mastery improvement were collected through experiment using one-group pretest-posttest design. Pretest and posttest were administered when the field testing was carried out. In the preliminary field testing, the pretest was ad- ministered before the multimedia teaching and learning process and posttest was ad- ministered after the multimedia teaching and learning process. The summary of the pretest and posttest data in the preliminary field testing is tabulated in Table 3. Table 1. Automotive electrical expert validation data Val.1 Val.2 Val.3 Val.4 Mean Mean 4.69 4.78 4.18 4.16 4.45 Table 2. Multimedia expert validation data Skor Val. 1 Skor Val. 2 Mean Mean 4.53 4.11 4.32 Table 3. Pretest and posttest data in preliminary field testing Average Improvement Pretest Posttest Points % 58.34 76.14 17.80 32.55 Research and Evaluation in Education Improving students' mastery on automotive electrical system... - 75 Dwi Widjanarko, Herminarto Sofyan, & Herman Dwi Surjono Based on the data in Table 3, it can be seen that the students’ mastery of automotive electrical systems before and after the appli- cation of the multimedia increases. The pre- test average score is 58.34 and the posttest average score is 76.14, so the improvement of the average score is 17.80 points, or it creates an increase up to 32.55%. The t test analysis shows that the calculated t is 1.979 and tabu- lated t is 3.169 (at 1% significance level) and 2.228 (at 5% significance level). This means that the calculated t value is less than the tabu- lated t value. There is no significant difference between the pretest average score and post- test average score. Although there is a mastery improvement of automotive electrical sys- tems, the improvement is not significant yet. Thus, the automotive electrical multimedia was not effective yet in increasing students’ mastery of automotive electrical systems. Based on the multimedia evaluation conduct- ed during the preliminary field testing, the multimedia was revised to accommodate stu- dents’ suggestions in order to gain the ease of operating the multimedia. After the revision of the multimedia, the main field testing was conducted. Pretest and posttest were also done in the main field testing. The tests were held in order to see the effectiveness of the multimedia during the main field testing. The summary of the pretest and posttest data in the main field testing is tabulated in Table 4. The mastery of the automotive elec- trical system before and after the application of the multimedia in the main field testing increased with the improved average score of 24.38 points or up to 64.89%. The t test analysis shows that the calculated t is 7.614 and tabulated t is 2.637 (at 1% significance level) and 1.989 (at 5% significance level). It means that the calculated t value is higher than the tabulated t value. There is a signifi- cant difference between the pretest average score and the posttest average score. There- fore, there is also students’ significant mastery improvement in the automotive electrical system in the main field testing. It seems that the automotive electrical multimedia is effect- ive in increasing students’ mastery of auto- motive electrical systems. Based on the multi- media evaluation which has been conducted during the main field testing, the multimedia was also revised to accommodate students’ suggestions. The last pretest and posttest were done in the operational field testing. The tests were held to see the effectiveness of the multi- media during the operational field testing. The summary of the pretest and posttest data in the operational field testing is tabulated in Table 5. The students’ mastery of automotive electrical systems before and after the appli- cation of the multimedia in the operational field testing increases with the average score improvement of 26.50 points or up to 77.36%. The t test analysis shows that the calculated t is 12.482 and the tabulated t is 2.576 (at 1% significance level) and 1.196 (at 5% significance level). This means that the calculated t value is higher than the tabulated t value. There is a significant difference be- tween the pretest average score and posttest average score. Thus, there is also a significant mastery improvement of automotive electrical systems in the operational field testing. It seems that the automotive electrical multi- media is effective in increasing students’ mastery of automotive electrical systems, and the learning objectives could be achieved. After the application of the automotive electrical multimedia in the field testing, there is a significant difference in the students’ mastery of automotive electrical systems before and after they use the multimedia. This means that the multimedia could effectively increase students’ mastery of automotive elec- trical systems. Table 4. Pretest and posttest data in main field testing Average Improvement Pretest Posttest Points % 46.30 70.68 24.38 64.89 Table 5. Pretest and posttest data in operational field testing Average Improvement Pretest Posttest Points % 43.78 70.28 26.50 77.36 Research and Evaluation in Education 76 − Volume 2, Number 1, June 2016 The improvement can be resulted from their habit by being more active in learning, exer- cising, and evaluating personally facilitated by the multimedia. Afolabi, Abidoye, and Afolabi (2012, p.6) state that effective utilization of appropriate instructional media is highly essential to improve teaching and learning of social studies in secondary schools. Improve- ment and better academic achievement can also be guaranteed through the use of in- structional media. In addition, if the standard of education has to be raised, instructional media should be used for the teaching and learning of social studies and other school subjects. The utilization way of computers in learning environments shows positive contri- butions of computer-based learning environ- ments to student learning (Deniz & Cakir, 2006, p.2) so the students’ academic achieve- ment can rise. The combination of computer- assisted instruction and collaborative work improves learning without a significant effect on attitude (Ragasa, 2008, p.8). The combin- ation of computer-assisted instruction and collaborative work improves learning without a significant effect on attitude (Ragasa, 2008, p.8). The improvement of achievement after the application of computer-based learning can be caused by the increase of learning motivation. Reeves (1998, p.2) states that computers as tutors have positive effects on learning as measured by standardized achieve- ment tests, are more motivating for students, and are accepted by more teachers than other technologies. Students could reach learning objectives in shorter time compared with conventional learning. The developed multi- media was interesting and able to help students master automotive electrical systems. This is based on students’ responses in the main field testing, stating that the developed multimedia could improve students’ motiv- ation. In the operational field testing, the multimedia also made their motivation to learn increase. A research conducted by Rosa and Preethi (2012, p.9) shows that students who are taught through multimedia instructional package can perform better than those who are taught through a common method of teaching. Multimedia enables learning through exploration, discovery, and experience. That role belong to the learning needs of students with multimedia, and the process of learning can become more goal-oriented, more partici- patory, flexible in time and space, unaffected by distances, and tailored to individual learn- ing styles. Multimedia enables learning to become fun and friendly, without fear of inadequacies or failure. In their research, they also found that there was a significant differ- ence of achievement between the experi- mental group taught with multimedia and the control group without multimedia. The con- clusion is that students learning with multi- media package got better result compared to those learning without multimedia package. Leow and Neo (2014, p.99) also find that an interactive multimedia learning bring signifi- cant improvement of students’ achievement and students’ activity and motivation in the learning processes. Some advantages of computer-based learning are that the students can choose their own way and speed, the program can be stoped at any time, the program can be repeated as often as the user wishes, the com- puter is not judgmental, the students can learn from their mistakes without embarrassment, it saves the teacher’s time, the students are more activated, and weak students are favoured (Schittek, Mattheos, Lyon, & Attström, 2001, p.99). In addition, Computer-Aided Learning (CAL) can provide innovative and interactive ways of presenting material and therefore should be used as an adjunct to conventional teaching or as a means of self-instruction. CAL can elicit positive responses from stu- dents and consequently motivate students to learn. A CAL program that is at least as effective as other methods of learning has several potential added value advantages (depending on how the program is designed and the students’ ease of access to the CAL modules): Students can learn at their own pace, CAL lessons can be reviewed several times, and computer-based modules can be used at convenient times when the student is free of distractions, alert, and ready to learn (Rosenberg, Grad, & Matear, 2003, p.531). Research and Evaluation in Education Improving students' mastery on automotive electrical system... - 77 Dwi Widjanarko, Herminarto Sofyan, & Herman Dwi Surjono Computer-Aided Instruction (CAI) is an application of computer in implementing instructions. It is an integration of software and hardware. Computer-based learning is especially effective for training people to use computer applications because the program can be integrated with the applications so that students can practice using the application as they learn (AlSultan, Lim, MatJafri, & Abdullah, 2006, p.29). Based on the afore- mentioned explanation, it can be concluded that utilizing computer technology in edu- cation brings many advantages especially for students. The learning objectives can be achieved through activating students in learn- ing processes. The learning process can be carried out in the classroom and also out of the classroom personally whenever and wher- ever. Thus, computer in education now be- comes a basic need to facilitate teaching and learning processes for teachers and students. Conclusion The validation which is conducted by automotive electrical and multimedia experts shows that the developed automotive elec- trical multimedia is valid, so it fulfills the requirements as an education multimedia. The application of multimedia in teaching and learning processes has significantly improved students’ mastery of automotive electrical systems. The improvement reached 32.55% in the preliminary field testing, 64.89% in the main field testing, and 77.36% in the oper- ational field testing. This indicates that automotive multimedia could increase the students’ understanding of automotive elec- trical systems significantly. Therefore, based on the research findings, the developed automotive electrical multimedia can be used as an instructional multimedia for teachers and as a learning tool for students to learn automotive electrical system seasily. References Afolabi, A.K., Abidoye, J.A., & Afolabi, A.F. (2012). Effect of instructional media on the academic achievement of stu- dents in social studies in junior second- ary schools. PNLA Quarterly, 77, 1-7. AlSultan, S., Lim, H.S., MatJafri, M. Z., & Abdullah, K. (2006). Developement of a computer aided instruction (CAI) package in remote sensing education. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Science, 34, 29-33. Arkorful, V. & Abaidoo, N. (2014). The role of e-learning, the advantages and dis- advantages of its adoption in higher education. International Journal of Education and Research, 2(2), 397-410. Azwar, S. (2003). Tes prestasi: Fungsi dan pengembangan pengukuran prestasi belajar (2 nd ed.) [Achievement test: Function and development of learning achieve- ment measurement (2 nd ed.)]. Yogya- karta: Pustaka Pelajar. Borg, W.R. & Gall, M.D. (1989). Educational research: An introduction (5 th ed.). New York, NY: Pearson Education. Deniz, H. & Cakir, H. (2006). Design principles for computer-assisted in- struction in histology education: An exploratory study. Journal of Science Education and Technology. doi: 10.1007/s10956-006-9031-5, 1-10. Eristi, S.D. (2008). The effectiveness of interactive instruction CD designed through the pre-school students. Journal of Theoretical and Applied Information Technology, 5, 832-839. Frey, B.A. & Sutton, J.M. (2010). A model for developing multimedia learning pro- jects. MERLOT Journal of Online Learning and Teaching, 6, 491-507. Garrand, T. (2006). Writing for multimedia and the web: A practical guide to content develop- ment for interactive media. Amsterdam: Elsevier. Ivers, K.S. & Barron, A.E. (2002). Multimedia projects in education designing, producing, and assessing (2 nd ed.). Connecticut, CT: Teacher Ideas Press, a division of Greenwood. Kayri, I., Gençoglu, M.T., & Kayri, M. (2012). The computer assisted education and its Research and Evaluation in Education 78 − Volume 2, Number 1, June 2016 effects on the academic success of students in the lighting technique and indoor installation project course. International Journal of Advances in Engineering & Technology, 2, 51-61. Leow, F.T. & Neo, M. (2014). Interactive multimedia learning: Innovating class- room education in a Malaysian uni- versity. TOJET: The Turkish Online Journal of Educational Technology . 13, 2, 99-110. Lu, C.H. & Cheng, S.F. (2012). Applying computer-based technology to instruct- ion for the effectiveness of teaching and learning. ARPN Journal of Science and Technology, 2(10), 1013-1017. Nazimuddin, S.K. (2014). Computer assisted instruction (CAI): A new approach in the field of education. International Journal of Scientific Engineering and Research (IJSER), 3(7), 185-188. Ragasa, C.Y. (2008). A comparison of computer-assisted instruction and the traditional method of teaching basic statistics. Journal of Statistics Education, 6(1), 1-10. Reddi, U.V. & Mishra, S. (2003). Educational multimedia: A handbook for teacher- developers. New Delhi: The common- wealth of Learning, Commonwealth Educational Media Centre for Asia. Reeves, T.C. (1998). The impact of media and technology in schools. A research report which is prepared for The Bertelsmann Foundation. University of Georgia. Rosa, M.C. & Preethi, C. (2012). Effective- ness of multimedia instructional pack- age for teaching marketing management among higher secondary school stu- dents. Education India Journal: A Quarterly Refereed Journal of Dialogues on Education, 1, 1-12. Rosenberg, H., Grad, H.A., & Matear, D.W. (2003). The effectiveness of computer- aided, self-instructional programs in dental education: A systematic review of the literature. Journal of Dental Education, 67, 524-532. Schittek, M., Mattheos, N., Lyon, H.C., & Attström, R. (2001). Computer assisted learning: A review. European Journal of Dental Education, 5, 93–100. Sidhu, M.S. (2010). Technology-assisted problem solving for engineering education: Interactive multimedia applications. New York, NY: Engineering Science Reference. Simkins, M., Cole, K., Tavalin, F., & Means, B. (2002). Increasing student learning through multimedia projects. Alexandria, VA: Association for Supervision and Curri- culum Development. Stemler, L.K. (1997). Educational character- istics of multimedia: A literature review. Journal of Educational Multimedia and Hypermedia, 6, 339-359. Widjanarko, D. & Abdurrahman. (2006). Peningkatan kemampuan menganalisis kerja sistem kelistrikan mobil dengan tugas model uraian terbatas [Develop- ment of analyzing ability of car elec- trical working system by employing restricted response task model]. Jurnal Pendidikan Teknik Mesin, 6, 59-64.