International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 06, 2021 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… Road Safety Education Courseware: A Study of Satisfaction and Learning Performance among Primary School Students in Malaysia https://doi.org/10.3991/ijim.v15i06.20637 Abdul Nasir Zulkifli(), Nur Fadziana Faisal Mohamed Universiti Utara Malaysia, Kedah, Malaysia abdulzulir@yandex.com Mustafa Moosa Qasim University of Basrah, Basrah, Iraq Nur Afiqah Abu Bakar Universiti Utara Malaysia, Kedah, Malaysia Abstract—There is a significant increase in road accident statistics in Malaysia and this reflects the culture formed among users related to road safety. In developing a positive road safety culture, road safety education should be introduced from an early age. The effort by the Ministry of Education Malaysia in implementing road safety education in primary schools since 2007 is a wise move towards the formation of that culture. As the road system becomes busier and more complex, children need to be educated with sufficient knowledge and skills to cope with increasingly challenging road situations. This paper introduced the ROSE courseware which incorporates multimedia, AR and VR technologies for the purpose of assisting teachers and students to understand and acquire skills related to road safety. A study was conducted among 30 primary school students in using the ROSE courseware for road safety education. The study provides an insight into the relationship between perceived ease of use, perceived usefulness, and enjoyment and students’ satisfaction in using the ROSE courseware utilising Pearson correlation and regression analyses. Regression analysis resultsindicate the significant relationships between perceived usefulness and enjoyment and students’ satisfaction in using the ROSE courseware. However, the relationship between perceived ease of use and satisfaction is notsignificant.Using paired sample t- test, this study also looked at the students' learning performance due to the intervention of the ROSE courseware for RSE. The outcomes showed that the mean scoredifference between the pre-test and post-test is significant. By using the ROSE courseware, thestudents gained significantly higher level of road safety knowledge which indicates that the learning performance among the participants has been enhanced through the implementation of the ROSE courseware. Keywords—Road Safety Education, ROSE courseware, Multimedia, Augmented Reality, Virtual Reality 48 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i06.20637 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… 1 Introduction In Malaysia, the statistics of road accidents show that the number of road accidents hasincreasedfrom 373,071 in 2005 to 533,875 in 2017 [27]. From 1974 to 2010, road accidents had been increasing at an average rate of 9.01% per annum in Malaysia [20]. Malaysia is estimated to have more than 20 road accidents deaths per 100,000 people in 2020 [20]. Human error is a major cause of a large number of these accidents which can be attributed to irresponsible and aggressive behaviour among road users in Malaysia [26]. Casualties can be reduced if road users have a well- developed road safety culture while they are young. The Malaysian government takes these accident statistics seriously and to address the road safety issue holistically, the government has taken several measures which include education, whereby the road safety education (RSE) program has been introduced in 2007 [60] and implemented in stages in the primary schools starting with year one [1]. RSE not only allows students to understand the road safety and regulations, but also to enable the students to experience the road environment in order to make RSE more effective. Besides, children need RSE-related knowledge as well as advanced cognitive skills to interact in road traffic situations. At present, RSE in primary schools across Malaysia is still adopting the conventional teaching and learning method without the use of computer or mobile- based application as a supplement. Among the first computer based RSE application that has been developed for Malaysian schools is ViSTREET [13]. It is a VR-based RSE learning application for school children. Version 2.0 of Virtual Reality Modelling Language (VRML) was used to generate VR-based scenarios and every recognized skill or problem is addressed by a distinct module. Users need to find a solution to the given problems by utilizing the problem manipulation area where all the scenarios created in ViSTREET were problem-based. This paper introduces the Road Safety Education (ROSE) courseware, an application that combines multimedia, virtual reality and augmented reality technologies in supporting teachers and students to understand and experience RSE. This courseware was developed based on the mixed learning environment whereby it includeslearning materialssuch as images, text, videos, animations, virtual environment and augmented reality scenes. The content of the ROSE courseware incorporates all the contents of the RSE skills prepared by the Ministry of Education of Malaysia. Multimedia blends graphics, text, audio, video and animation and it is a technology that provides opportunities to escape from conventional teaching. Multimedia has been proven to be able to increase the effectiveness of learning [18], create student- centred learning [36], improve the level of understanding [65], create fun learning atmosphere [2], enable users to observe behaviour and engage in the learning process [48] and enhance learning motivation [69]. Fig. 1 shows some snapshots of multimedia related learning materials available in the ROSE courseware. iJIM ‒ Vol. 15, No. 06, 2021 49 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… Fig. 1. Multimedia related learning materials in the ROSE courseware Virtual reality enables users to experience the sensation of presence in different physical places [6]. In teaching and learning in schools, VR is able to provide a new way of reaching more students [40][56][5], facilitates themin experiencing learning [63] [38] [61] [7] and allows them to practice risk-free procedures through simulations [57] [54] [33] [32]. For ROSE courseware, VR has been used to simulate zebra and pedestrian bridge crossings as shown in Fig. 2. Fig. 2. Virtual reality related learning materials in the ROSE courseware Finally, augmented reality (AR) allows digital objects to be overlayed onto the real environment allowing users to interact with them [37]. In learning, AR provides many benefits which include; enhancing fun and entertainment, [21] [44], in situ interactive visualization [4] [50] [29], and improved learning performance and motivation [21] [9] [62] [10] [8] [55]. For ROSE courseware, marker-based AR has been implemented and it has been used for the purpose of teaching students about road signs. Fig. 3 shows samples of the road sign markers and a video related to the stop sign. Fig. 3. Augmented reality related learning materials in the ROSE courseware 50 http://www.i-jim.org Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… User and learning performance evaluations have been conducted among 30 students of a primary school. User evaluation was conducted in determining the students’ perception towards using the ROSE course in terms of perceived ease of use, perceived usefulness, enjoyment and satisfaction and also to determine the relationships between perceived ease of use, perceived usefulness, and enjoyment (independent variables) and satisfaction (dependent variable). Meanwhile, learning performance evaluation was conducted in determining the achievement of the students with and without the intervention of the ROSE courseware. 2 Method 2.1 Research model The ROSE courseware is intended in providing information about RSE interactively utilizing multimedia, AR and VR to primary school students. Thus, it is important to understand the relations between the variables whenusing the ROSE coursewarefor RSE. The model for this study is shown in Fig.4. Fig. 4. Model of Study 2.2 Research hypotheses The following hypotheses have been formulated for this study. Hypothesis1: There is a relationship between Perceived Ease of Use and Satisfaction. Hypothesis2: There is a relationship between Perceived Usefulness and Satisfaction. Hypothesis3: There is a relationship between Enjoyment and Satisfaction. Hypothesis4: There is a significant difference between the pre-test and post-test mean scores. 2.3 Research design Since this study involves user and learning performance evaluations, it has been designed to utilise hypotheses testing, correlational and one group pre-test and post- test. Quantitative data has been utilised in this study through the use of questionnaires for the user evaluation and RSE-related questions for the learning performance iJIM ‒ Vol. 15, No. 06, 2021 51 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… evaluation. The user evaluation determined the participants’ perceptions towards using the ROSE courseware and also the relationships between the independent variables and the dependant variable among a sample of 30 primary school students. The learning performance evaluation was carried outin determining the students' learning performance with and without the intervention of the ROSE courseware. 2.4 Participants Both evaluations have been conducted among a sample of 30 students of a primary school in Sintok, Kedah, Malaysia. The sample has been selected utilizing purposive sampling, a techniqueof taking sample based on criteria set by the researchers [59]. The number of samples was 30% out of 100 which was the total number of year four students in this school and this fulfills the minimum number of samplesas suggested by [14]. 2.5 Measurements The measurements that were used in the user evaluation include perceived ease of use, perceived usefulness, enjoyment and satisfaction. Perceived ease of use is the degree in which one thinks that using technology will be effortless [16]. According to the Technology Acceptance Model, one of the determinants of one’s intention to use an applicationis perceived ease of use [17]. Since the ROSE courseware incorporates multimedia, AR and VR technologies, perceived ease of useis vital to the primary school students so that they are able to use the ROSE courseware easily. Perceived usefulness is one's level of confidence by using certain technology will improve one's performance [66]. [16] concurs with [66] by defining perceived usefulness as the degree where one believes that utilising a technology will improve his/her performance. The ROSE courseware can only be accepted by the primary school students when they perceive that its use is going to enhance their performance in learning about road safety. Enjoyment is defined as the extent to which an activity is considered as giving pleasure and joy in its rights, apart from the consequences of performance [67]. Enjoyment is a good feeling that can reduce tension and increase learning motivation [66]. Satisfaction is defined as the users’ response after using a system which makes the users feel positive [47]. Satisfaction in learning is achieved when the students' needs and desires are met in the learning process [31]. Based on previous studies, multimedia, AR and VR are able to provide satisfaction [35] [34] [41] [19] [68] [39] [28] [70] [11] [55]. 2.6 Instrument There are two types of instruments used in this study namely; user evaluation instrument and learning performance instrument. The user evaluation instrument is a set of questionnaires which comprises of measurements that include; perceived ease of use, perceived usefulness, enjoyment and satisfaction. The items for perceived ease of use were cited from [25], perceived usefulness were cited from [16], enjoyment 52 http://www.i-jim.org Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… were cited from [66] and lastly satisfaction were cited from [12]. The user evaluation instrument comprises of demographic data of participants and participant’s perceptions towards using the ROSE courseware. It uses a five-point Likert scale ranging from 1 to 5 (strongly disagree - strongly agree). Meanwhile, the learning performance instrument consists of a set of 15 multiple choice questions related to the RSE contents. 2.7 Procedure All the 30 participants were gathered in a computer lab. Firstly, the learning performance instruments were distributed among the participants. A predetermined identification has been assigned to each participant’s learning performance instrument for the purpose of data analysis and also to ensure that all the pre-test data was collected anonymously. The participants were given 30 minutes in answering the questions. Then, the participants were given 10 minutes to rest before proceeding with a short briefing on the function and interfaces of the ROSE courseware. Next, they were given an hour to be familiar with using the ROSE courseware. Later, the participants were handed with a set of questionnaires and they have to complete the user evaluation within 30 minutes. Once finished, they were given an hour to rest. For the final task, they were asked to return to the lab to complete the learning performance evaluation. The participants were again handed the learning performance instrument and they were again allocated 30 minutes to answer the questions. All the post-test data was again collected anonymously. 3 Results 3.1 Demographic characteristic Table 1 shows the participants’ demographic data. The participantscomprised of 13 males and 17 females. They were 10 years old. The participants comprised of 25 Malays, 3 Chinese, 2 Indians, and 1 Siamese. Table 1. Respondents’ Demographic Data Respondents’ Profile Frequency Percentage (%) Gender 1. Male 13 43 2. Female 17 57 Race 1. Malay 24 83 2. Chinese 3 10 3. Indian 2 6.7 4. Others 1 3.3 iJIM ‒ Vol. 15, No. 06, 2021 53 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… 3.2 Validity and reliability Since all the measurements and items were adapted from previous studies, they have undergone the proper validation process and are deemed valid. Reliability analysis was performed in ensuring the items’ stability or consistency of an instrument [58]. SPSS version 22.0 was used to calculate the Cronbach alpha for all the measurements. Based on Table 2, the Cronbach alphas forperceived ease of use, perceived usefulness, enjoyment and satisfaction are 0.789, 0.719, 0.747, and 0.835 respectively. They are considered reliable sincethe Cronbach alphas for all measurementsare greater than 0.7 [46] [23]. Table 2. Cronbach Alpha Measurement Number of Items Cronbach Alpha Perceived ease of use 4 0.789 Perceived usefulness 3 0.719 Enjoyment 4 0.747 Satisfaction 6 0.835 3.3 Descriptive statistics Table 3 shows the mean for all the measurements and items. A5-point Likert scale was utilised in the instrument for this study to determine the mean. Since Likert scale is an ordinal scale and it should be converted to numerical scale before interpreting the mean as recommended by [52]. In converting from Likert scale to numerical scale, Sugiyono equation [64] was applied. The numerical scale has to be reclassified into strongly disagree, disagree, agree and strongly agree. The Sugiyono equation is as follows: RS = (m - n) / b = (5-1)/4 = 1 Note: RS - Score range m - highestpoint on Likert scale n - lowestpoint on Likert scale b - classification number The result from the equation was then used to produce a numerical scale consisting of strongly disagree with mean ranges from 1 - 1.99, disagree with mean ranges from 2 - 2.99, agree with mean ranges from 3 - 3.99, and strongly agree with mean ranges from 4 – 5. The results of the descriptive statistics analysis based on the numerical scale showed that perceived ease of use has mean of 4.21 (strongly agree), perceived usefulness has mean of 4.24 (strongly agree), enjoyment has mean of 4.15 (strongly agree), and satisfaction has mean of 4.22 (strongly agree). 54 http://www.i-jim.org Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… Table 3. Mean Measurement Mean Perceived Ease of Use 4.21 It was easy to use the ROSE courseware 4.23 ROSE courseware wassuitable for learning about road safety 4.27 ROSE courseware was easy to use when you are alone 4.07 Steps in using the ROSE courseware were easy to remember 4.27 Perceived Usefulness 4.24 The ROSE courseware was useful to me. 4.13 The ROSE courseware enabled me to learn about road safety 4.33 The ROSE courseware saved me time in learning about road safety 4.27 Enjoyment 4.15 I really like and enjoy using the ROSE courseware in learning about road safety 4.23 The ROSE courseware made me deeply enjoyed learning 4.03 I enjoyed learning road safety using the ROSE courseware 4.07 The ROSE courseware cultivates the interest in learning about road safety 4.27 Satisfaction 4.22 I was satisfied with ROSE courseware learning experience 4.17 I was satisfied with the effectiveness of the overall learning 4.33 I was satisfied with the methods of learning provided by the ROSE courseware 4.10 I was satisfied with thelearning environmentof the ROSE courseware. 4.30 I found that the ROSE courseware met my needs 4.20 3.4 Correlation The correlation analysis is used in determiningthe relationship between the independent and dependant variables. Table 4 shows the outcome of the Pearson correlation analysis which shows that thereare positive correlations between all the independent variables and the dependent variable. The correlation valuesfor perceived ease of use, perceived usefulness, and enjoyment are .954, .949, and .928 respectively. Since the values are greater than 0.70, the correlation among the variables is very strong [49]. Table 4. Pearson Correlation Variables Satisfaction Perceived ease of use Perceived Usefulness Enjoyment Satisfaction 1 Perceived ease of use .954** 1 Perceived Usefulness .949** .966** 1 Enjoyment .928** .928** .918 1 Note: Correlation is significant at the 0.01 level (1-tailed) ** 3.5 Regression The purpose of the regression analysis is to estimate the relationships between the independent and the dependent variablesin order to verify the hypotheses and the results of the regression analysis are shown in Table 5. The R2 value is 0.976 which iJIM ‒ Vol. 15, No. 06, 2021 55 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… reflects the students’ satisfaction towards the use of the ROSE courseware. The independent variables are the predictors in this analysis. The R2 value for this multiple regression is 0.976 which means that 98% of satisfaction variation is related to variation in perceived ease of use, perceived usefulness and enjoyment; leaving 2% unexplained. According to [15], the acceptable p-value should not more than 0.05 and the t-value should be greater than 1.645. Table 5. Regression Variable Beta Std. Error t-value Sig (p-value) Perceived ease of use 0.34 0.18 2.977 0.06 Perceived usefulness 0.27 0.13 2.162 0.04* Enjoyment 0.21 0.09 2.338 0.03* **Significance level; p < 0.01 * Significance level; p < 0.05 Dependent Variable: Satisfaction N=30; R Square = 0.976; Adjusted R Square = 0.969; F = (6.23) 153.706 3.6 Learning performance evaluation A one group pre-test and post-test was implemented to observe the participants’ learning performance due to the intervention of the ROSE Courseware for RSE. The questions for this evaluation were based on the RSE contents. Fifteen multiple choice questions were distributed among 30 students and a predetermined identification has been assigned to each respondent’s questionnaire booklet for the purpose of data analysis. The students were allocated 30 minutes to answer the questions. All the data was collected anonymously. In assessing the learning performance of the respondents, paired-sample t-test was applied. The pre-test and post-test mean scores which represent the participants’ learning performances are shown in Table 6. The mean scores increased from 17.97 to 23.20. This indicates that the students were able to improve their learning performance and perform better compared to the previous test. The questionsfor the pre and posttestswerethe same; however, the questions’ order has been changed. Table 6. Learning Performance Evaluation Mean Number of participants Std. dev. Std. Error Mean Pre-test 17.97 30 1.691 .309 Post-test 23.20 30 1.215 .222 Table 7 presentsthe paired-sample t-test results. The pre-test mean score represents the participants’ background knowledge pertaining to RSE which has been gained from various sources including newspapers, social media, documentaries, YouTube, TV, magazines, internet, and many others. The post-test mean score represents the participants' learning performance through their background knowledge coupled with knowledge gained through the use of the ROSE software. Since the significant value is 0.000 which is significant at 0.01, the difference between the participants’ pre-test 56 http://www.i-jim.org Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… and post-test mean scores is significant.By comparing the outcomes of pre-test and post-test, participants who used the ROSE courseware gained significantly higher level of knowledge. Generally, the learning performance for the RSE among the participants could be enhanced through the implementation of the ROSE courseware. Table 7. Paired Sample t-test Learning Evaluation Paired Differences t df Sig.(2- tailed) Mean Std. Deviation Std.Error Mean 95% Confidence Interval of the Difference Mean Std.Dev. Std.Error Mean Lower Upper Pre-test and Post-test -5.233 1.591 .290 -5.827 -4.639 -18.021 29 .000 3.7 Hypothesis testing Hypothesis testing has been conducted to examine the relationships between the independent variables and satisfaction. For that purpose, the following hypotheses have been formulated. Hypothesis1: There is a relationship between Perceived Ease of Use and Satisfaction. Hypothesis2: There is a relationship between Perceived Usefulness and Satisfaction. Hypothesis3: There is a relationship between Enjoyment and Satisfaction. Hypothesis4: There is a significant difference between the pre-test and post-test mean scores. As hypothesized in H1, there is a relationship betweenperceived ease of use and satisfaction. As shown in Table 5, perceived ease of use has nosignificant relationship with satisfaction with p = 0.06. Since the p-value is higher than 0.05, the hypothesis is rejected. As hypothesized in H2, there is a relationship between perceived usefulness and satisfaction. Table 5 shows that perceived usefulness has significant relationship with satisfaction with p = 0.04. Since the p-value is 0.04 which is smaller than 0.05, the hypothesis is supported. As hypothesized in H3, thereis a relationship between enjoyment and satisfaction in using the ROSE courseware. Table 5 shows that enjoyment has significant relationship with satisfaction with p = 0.03. Since the p- value is 0.03 which is smaller than 0.05, the hypothesis is supported. As hypothesized in H4, there is a significant difference between the pre-test and post-test mean scores. Based on Table 7, thep-value is 0.000 which indicates that this hypothesis is supported. 4 Conclusion Road accident statistics continue to show an increase in terms of deaths and injuries among road users in Malaysia. Therefore, road users need to be educated from an early age in relation to road safety rules and regulations. Nowadays, children iJIM ‒ Vol. 15, No. 06, 2021 57 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… need to be inculcated with sufficient knowledge and skills in coping with the complex road situations so that they can develop a road safety culture within themselves in preparation for today's increasingly challenging road traffic spectrum. This paper introduced the ROSE courseware which incorporates multimedia, AR and VR technologies for the purpose of assisting teachers and students to understand and acquire skills related to RSE. This paper also discusses the participants’ perceptions whereby they strongly agreed on the perceived ease of use, perceived usefulness, enjoyment and satisfaction of using the ROSE courseware. It also gives an overview of the relationship between the independent variables and satisfaction in using the ROSE courseware among the students. The results support empirically the statistically significant relationships between perceived usefulness and enjoyment and satisfaction in using the ROSE courseware. Previous empirical studies have shown that perceived usefulness is one of the key determinants of the use of a particular technology [22] [3] [43] [55] [70] [71]. Based on previous studies also, multimedia, AR and VR technologies are able to provide the feeling of enjoyment [53] [8] [24] [30] [21] [45]. Nevertheless, the outcome of this study indicates that the relationship between perceived ease of use and satisfaction is not significant. The results of thisstudy are similar to the studies by [51] and [22], where AR and VR are not acknowledged by new users as easy to use since they need to be familiar and comfortable with the new learning environment. In order to observe the performance of students in learning with and without using the ROSE courseware, a one group pre-test and post-test was conducted. The results of the paired sample t-test showed that the difference in the participants’ mean scores between the pre-test and post-test is significant. The participants who have used the ROSE courseware gained significantly higher level of RSE knowledge which indicates that the learning performance among the participants has been enhanced through the implementation of the ROSE courseware. The ROSE courseware is not intended to replace the current approach of teaching RSE in schools. Instead, it is just an additional learning tool that can be used by teachers and students inside or outside the classroom to increase RSE-related understanding. It not only provides information that can improve knowledge but also can improve students' skills through the utilisation of AR and VR technologies. 5 Acknowledgement We would like to thank the Awang Had Salleh Graduate School, Universiti Utara Malaysia for supporting this research and to Nur Afiqah Abu Bakar who has helped us in completing this research. 6 References [1] Abu Bakar, N. A., Zulkifli, A. N., & Mohamed, N. F. F. (2011, September). The use of multimedia, Augmented Reality (AR) and Virtual Environment (VE) in enhancing children's understanding of road safety. In 2011 IEEE conference on open systems. IEEE. 149-154. https://doi.org/10.1109/icos.2011.6079288 58 http://www.i-jim.org https://doi.org/10.1109/icos.2011.6079288 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… [2] Aisyah, R., Zakiyah, I. A., Farida, I., & Ramdhani, M. A. (2017, September). Learning Crude Oil by Using Scientific Literacy Comics. In Journal of Physics: Conference Series. IOP Publishing. 895(1): 012011.https://doi.org/10.1088/1742-6596/895/1/012011 [3] Al-Busaidi, K. A. (2013). An empirical investigation linking learners’ adoption of blended learning to their intention of full e-learning. Behaviour & Information Technology. 32(11): 1168-1176. https://doi.org/10.1080/0144929x.2013.774047 [4] Aljojo, N., Munshi, A., Zainol, A., Al-Amri, R., Al-Aqeel, A., Al-khaldi, M., ... &Qadah, J. (2020). Lens application: Mobile application using augmented reality. International Journal of Interactive Mobile Technologies. 14(2): 160-177. https://doi.org/10.3991/ ijim.v14i02.11726 [5] Bell, J. T., &Fogler, H. C. (2004, March). The application of virtual reality to (chemical engineering) education. In IEEE Virtual Reality 2004. IEEE. 217-218. https://doi.org/10.1109/vr.2004.1310077 [6] Benoit, M., Guerchouche, R., Petit, P. D., Chapoulie, E., Manera, V., Chaurasia, G., ... & Robert, P. (2015). Is it possible to use highly realistic virtual reality in the elderly? A feasibility study with image-based rendering. Neuropsychiatric disease and treatment. 11: 557. https://doi.org/10.2147/ndt.s73179 [7] Bricken, M. (1991). Virtual reality learning environments: potentials and challenges. AcmSiggraph Computer Graphics. 25(3): 178-184. https://doi.org/10.1145/126640. 126657 [8] Cai, S., Chiang, F. K., Sun, Y., Lin, C., & Lee, J. J. (2017). Applications of augmented reality-based natural interactive learning in magnetic field instruction. Interactive Learning Environments. 25(6): 778-791. https://doi.org/10.1080/10494820.2016.1181094 [9] Cen, L., Ruta, D., Al Qassem, L. M. M. S., & Ng, J. (2019). Augmented Immersive Reality (AIR) for Improved Learning Performance: A Quantitative Evaluation. IEEE Transactions on Learning Technologies. https://doi.org/10.1109/tlt.2019.2937525 [10] Chang, S. C., & Hwang, G. J. (2018). Impacts of an augmented reality-based flipped learning guiding approach on students’ scientific project performance and perceptions. Computers & Education. 125: 226-239. https://doi.org/10.1016/j.compedu.2018. 06.007 [11] Chen, Y. F., Luo, Y. Z., Fang, X., & Shieh, C. J. (2018). Effects of the application of computer multimedia teaching to automobile vocational education on students’ learning satisfaction and learning outcome. Eurasia Journal of Mathematics, Science and Technology Education. 14(7): 3293-3300. https://doi.org/10.29333/ejmste/91245 [12] Chou, S. W., & Liu, C. H. (2005). Learning effectiveness in a Web‐based virtual learning environment: a learner control perspective. Journal of computer assisted learning. 21(1): 65-76. https://doi.org/10.1111/j.1365-2729.2005.00114.x [13] Chuah, M., & Chen, C. J. (2008). Unleashing the Potentials of Desktop Virtual Reality as an Educational Tool: A Look into the Design and Development Process of ViSTREET. In Proc. 2nd International Malaysian Educational Technology Convention (IMETC2008). 81- 86. [14] Coakes, S. J. and L. G. Steed (2003). SPSS Analysis without anguish, version 11.0 for windows, John Wiley & Sons Australia. [15] Cohen, B. H. (2008). Explaining psychological statistics. John Wiley & Sons. [16] Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly. 319-340. https://doi.org/10.2307/249008 [17] Davis, F. D., Bagozzi, R. P., &Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management science. 35(8): 982- 1003. https://doi.org/10.1287/mnsc.35.8.982 iJIM ‒ Vol. 15, No. 06, 2021 59 https://doi.org/10.1088/1742-6596/895/1/012011 https://doi.org/10.1080/0144929x.2013.774047 http://www.i-jim.org/ http://www.i-jim.org/ https://doi.org/10.3991/ijim.v14i02.11726 https://doi.org/10.3991/ijim.v14i02.11726 https://doi.org/10.1109/vr.2004.1310077 https://doi.org/10.2147/ndt.s73179 https://doi.org/10.1145/126640.126657 https://doi.org/10.1145/126640.126657 https://doi.org/10.1080/10494820.2016.1181094 https://doi.org/10.1109/tlt.2019.2937525 https://doi.org/10.1016/j.compedu.2018.06.007 https://doi.org/10.1016/j.compedu.2018.06.007 https://doi.org/10.29333/ejmste/91245 https://doi.org/10.1111/j.1365-2729.2005.00114.x https://doi.org/10.2307/249008 https://doi.org/10.1287/mnsc.35.8.982 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… [18] Fatdha, T. S., &Wahyuni, D. S. (2017, October). Designing English Learning Media for Starter Learners at Elementary Schools based on Interactive Multimedia. In International Conference of Applied Science on Engineering, Business, Linguistics and Information Technology. ICo-ASCNITECH. 204-209. [19] Freina, L., & Ott, M. (2015, April). A literature review on immersive virtual reality in education: state of the art and perspectives. In The international scientific conference elearning and software for education. 1(133): 10-1007. [20] Ghadiri, S. M. R., Prasetijo, J., Sadullah, A. F., Hoseinpour, M., & S. Sahranavard (2013). Intelligent speed adaptation: Preliminary result of on-road study in Penang Malaysia. IATSS Research. 36: 106-114. https://doi.org/10.1016/j.iatssr.2012.08.001 [21] Gopalan, V., Zulkifli, A. N., & Abu Bakar, J. A. (2016). A Study of Students’ Motivation Based on Ease of Use, Engaging, Enjoyment and Fun Using the Augmented Reality Science Textbook. Revista de la Facultad de Ingeniería U.C.V. 31(5): 27-35. https://doi.org/10.21311/002.31.5.04 [22] Gopalan, V., Zulkifli, A. N., & Abu Bakar, J. A. (2016, August). A study of students’ motivation using the augmented reality science textbook. In AIP Conference Proceedings. AIP Publishing LLC. 1761(1): 020040. https://doi.org/10.1063/1.4960880 [23] Hair, J. F. (2006). Multivariate data analysis. Prentice Hall, New Jersey [24] Ibáñez, M. B., Di Serio, Á., Villarán, D., &Kloos, C. D. (2014). Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness. Computers & Education. 71: 1-13. https://doi.org/10.1016/j. compedu.2013.09.004 [25] Idris, H., Zulkifli, A. N., &Yusoff, M. F. (2019). MARA: A Mobile Augmented Reality Approach to Advertising for Printed Media Microenterprise. International Journal of Recent Technology and Engineering. 7(6S2): 126-131. [26] Ismail, R., Ibrahim, N., Rad, A. Z., &Borhanuddin, B. (2009). Angry thoughts and aggressive behavior among Malaysian driver: A preliminary study to test model of accident involvement. European Journal of Social Sciences. 10(2): 273-281. [27] JabatanKeselamatan Jalan Raya Malaysia (JKJR). (2018). Bukustatistikkeselamatanjalanraya Malaysia 2018 (Malaysia: IbuPejabat JKJR). [28] Joo-Nagata, J., Abad, F. M., Giner, J. G. B., & García-Peñalvo, F. J. (2017). Augmented reality and pedestrian navigation through its implementation in m-learning and e-learning: Evaluation of an educational program in Chile. Computers & Education. 111: 1-17. https://doi.org/10.1016/j.compedu.2017.04.003 [29] Kerawalla, L., Luckin, R., Seljeflot, S., & Woolard, A. (2006). “Making it real”: exploring the potential of augmented reality for teaching primary school science. Virtual reality. 10(3-4): 163-174. https://doi.org/10.1007/s10055-006-0036-4 [30] Kim, M. J., Chung, N., Lee, C. K., &Preis, M. W. (2015). Motivations and use context in mobile tourism shopping: Applying contingency and task–technology fit theories. International Journal of Tourism Research. 17(1): 13-24. https://doi.org/10.1002/ jtr.1957 [31] Kim, Y., Son, Y., & Han, S. (2016). What is the Role of TV Commercials in the Trans- Media Era? Indian Journal of Science and Technology. 9(41): 1-10. https://doi.org/10.17485/ijst/2016/v9i41/103846 [32] Kustandi, C., Fadhillah, D., Situmorang, R., Prawiladilaga, D., &Hartati, S. (2020). VR Use in Online Learning for Higher Education in Indonesia. International Journal of Interactive Mobile Technologies. 14(1): 31-47. https://doi.org/10.3991/ijim.v14i01. 11337 60 http://www.i-jim.org https://doi.org/10.1016/j.iatssr.2012.08.001 https://doi.org/10.21311/002.31.5.04 https://doi.org/10.1063/1.4960880 https://doi.org/10.1016/j.compedu.2013.09.004 https://doi.org/10.1016/j.compedu.2013.09.004 https://doi.org/10.1016/j.compedu.2017.04.003 https://doi.org/10.1007/s10055-006-0036-4 https://doi.org/10.1002/jtr.1957 https://doi.org/10.1002/jtr.1957 https://doi.org/10.17485/ijst/2016/v9i41/103846 http://www.i-jim.org/ http://www.i-jim.org/ https://doi.org/10.3991/ijim.v14i01.11337 https://doi.org/10.3991/ijim.v14i01.11337 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… [33] Lamblin, G., Thiberville, G., Druette, L., Moret, S., Couraud, S., Martin, X., ... &Chene, G. (2020). Virtual reality simulation to enhance laparoscopic salpingectomy skills. Journal of Gynecology Obstetrics and Human Reproduction. 49(3): 101685. https://doi.org/10. 1016/j.jogoh.2020.101685 [34] Lee, E. A. L., Wong, K. W., & Fung, C. C. (2010). Learning with virtual reality: Its effects on students with different learning styles. In Transactions on edutainment IV. Springer, Berlin, Heidelberg. 79-90. https://doi.org/10.1007/978-3-642-14484-4_8 [35] Liaw, S. S. (2008). Investigating students’ perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system. Computers & education. 51(2): 864-873. https://doi.org/10.1016/j.compedu.2007.09.005 [36] Liliasari, M., Supriyanti, S., & Hana, M. N. (2016). Students’ Creative Thinking Enhancement By Using Interactive Multimedia of Redox Reaction. Jurnal Pengajaran MIPA. 21(1): 30-34. [37] Liu, W., Cheok, A. D., Mei-Ling, C. L., & Theng, Y. L. (2007, September). Mixed reality classroom: learning from entertainment. In Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts. 65-72. https://doi.org/10. 1145/1306813.1306833 [38] Man, J., Guo, F., & Ma, C. (2020, April). Innovative Analysis of Higher Vocational Education Model Based on Virtual Reality Technology. In Journal of Physics: Conference Series. IOP Publishing. 1533(2): 022097.https://doi.org/10.1088/1742- 6596/1533/2/022097 [39] Mesia, N. S., Sanz, C., & Gorga, G. (2016, October). Augmented reality for programming teaching. Student satisfaction analysis. In 2016 international conference on collaboration technologies and systems (cts). IEEE. 165-171.https://doi.org/10.1109/cts.2016. 0045 [40] Misak, J. (2018). A (Virtual) Bridge Not Too Far: Teaching Narrative Sense of Place with Virtual Reality. Computers and Composition. 50: 39-52. https://doi.org/10.1016/j. compcom.2018.07.007 [41] Mohamed Noor, N., Zulkifli, A. N., & Siraj, F. (2014). Islamic Sex Education (ISE) conceptual model of cognitive theories–the findings. Jurnal Teknologi. 68(2): 13-18. https://doi.org/10.11113/jt.v68.2904 [42] Mohd Jelani, N. A., Zulkifli, A. N., Ismail, S., &Yusoff, M. F. (2017, October). Taekwondo trainees’ satisfaction towards using the virtual taekwondo training environment prototype. In AIP Conference Proceedings. AIP Publishing LLC. 1891(1): 020099. https://doi.org/10.1063/1.5005432 [43] Mohd Jelani, N.A., Zulkifli, A.N., Ismail, S., &Yusoff M.F. (2019). A Study of Trainees Satisfaction using the Virtual Taekwondo Training Environment (VT2 E) Prototype. International Journal of Innovative Technology and Exploring Engineering. 8(5s): 9-15. https://doi.org/10.1063/1.5005432 [44] Nazaruddin, M. A., & Efendi, M. (2018). The book of pop up augmented reality to increase focus and object recognition capabilities for children with autism. Journal of ICSAR. 2(1): 9-14. https://doi.org/10.17977/um005v2i12018p009 [45] Ngwira, F. F., Kamwaza, M., Rashid, S., Boby, G., &Kadzakumanja, G. (2019). Medical and allied health students’ self-regulated learning: The interplay between motivational beliefs and learning strategies. J Contemp Med Edu: 9(1). https://doi.org/10. 5455/jcme.20181013060753 [46] Nunnally, J. C. (1978). Psychometric theory. McGraw-Hill, New York. [47] Oh, B. G., & Lee, S. H. (2015). Impact of Factors Related to Taekwondo Participants' Exercise Experience on Their Satisfaction with Acceptance of WOM Information, and iJIM ‒ Vol. 15, No. 06, 2021 61 https://doi.org/10.1016/j.jogoh.2020.101685 https://doi.org/10.1016/j.jogoh.2020.101685 https://doi.org/10.1007/978-3-642-14484-4_8 https://doi.org/10.1016/j.compedu.2007.09.005 https://doi.org/10.1145/1306813.1306833 https://doi.org/10.1145/1306813.1306833 https://doi.org/10.1088/1742-6596/1533/2/022097 https://doi.org/10.1088/1742-6596/1533/2/022097 https://doi.org/10.1109/cts.2016.0045 https://doi.org/10.1109/cts.2016.0045 https://doi.org/10.1016/j.compcom.2018.07.007 https://doi.org/10.1016/j.compcom.2018.07.007 https://doi.org/10.11113/jt.v68.2904 https://doi.org/10.1063/1.5005432 https://doi.org/10.1063/1.5005432 https://doi.org/10.17977/um005v2i12018p009 https://doi.org/10.5455/jcme.20181013060753 https://doi.org/10.5455/jcme.20181013060753 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… Spread by WOM. Indian Journal of Science and Technology. 8: 46. https://doi.org/10. 17485/ijst/2015/v8is8/64711 [48] Omar, S. B., Choo, K. A.,&Bidin, A. (2020). The Influence of Multimedia with Autistic Learners from the Teachers Perception. International Journal of Interactive Mobile Technologies. 14(14): 52-63. https://doi.org/10.3991/ijim.v14i14.12943 [49] Pallant, J. (2007). SPSS survival manual: A step by step guide to data analysis using SPSS for windows. Singapore: Markono Print Media Pte Ltd. https://doi.org/10.4324/ 9781003117452 [50] Peterson, C. N., Tavana, S. Z., Akinleye, O. P., Johnson, W. H., & Berkmen, M. B. (2020). An idea to explore: Use of augmented reality for teaching three‐dimensional biomolecular structures. Biochemistry and Molecular Biology Education. https://doi.org/10.1002/ bmb.21341 [51] Pribeanu, C. (2014). Measuring the Effects of Usability Issues Affecting an Enjoyable Learning Experience-A Path Analysis Approach. Journal of Applied Quantitative Methods. 9(3): 14-23. [52] Qasim, M. M., Ahmad, M., Omar, M., Zulkifli, A. N., & Abu Bakar, J. A. (2018b). A process for developing an instrument to measure the persuasion perspectives of parents using PMCOM app. In AIP Conference Proceedings. AIP Publishing. 2016(1): 020119. https://doi.org/10.1063/1.5055521 [53] Rohaya, D., Rambli, A., Matcha, W., Sulaiman, S., &Nayan, M. Y. (2012). Design and development of an interactive augmented reality edutainment storybook for preschool. IERI Procedia. 2: 802-807. https://doi.org/10.1016/j.ieri.2012.06.174 [54] Rudarakanchana, N., Van Herzeele, I., Desender, L., & Cheshire, N. J. (2015). Virtual reality simulation for the optimization of endovascular procedures: current perspectives. Vascular health and risk management. 11: 195. https://doi.org/10.2147/vhrm. s46194 [55] Rusli, F. N., Zulkifli, A., bin Saad, M., &Yussop, Y. M. (2019). A Study of Students’ Motivation in Using the Mobile Arc Welding Learning App. International Journal of Interactive Mobile Technologies. 13(10): 89-105. https://doi.org/10.3991/ijim. v13i10.11305 [56] Sampaio, A. Z. (2012). Virtual reality technology applied in teaching and research in civil engineering education. Journal of Information Technology and Application un Education. 1(4): 152-163. [57] Schwebel, D. C., Davis, A. L., & O’Neal, E. E. (2012). Child pedestrian injury: A review of behavioral risks and preventive strategies. American journal of lifestyle medicine. 6(4): 292-302. https://doi.org/10.1177/0885066611404876 [58] Sekaran U. (2003). Research methods for business: A skill-building approach, John Wiley & Sons, Inc. [59] Sekaran, U., & Bougie, R. (2006). Research methods for business: A skill building approach (5th ed.). John Wiley and Sons, Chichester. [60] Shahid, S., Minhans, A., Puan, O. C., Hasan, S. A., & Ismail, T. (2015). Spatial and temporal pattern of road accidents and casualties in Peninsular Malaysia. Jurnal Teknologi. 76(14). https://doi.org/10.11113/jt.v76.5843 [61] Shim, K. C., Park, J. S., Kim, H. S., Kim, J. H., Park, Y. C., & Ryu, H. I. (2003). Application of virtual reality technology in biology education. Journal of Biological education. 37(2): 71-74. https://doi.org/10.1080/00219266.2003.9655854 [62] Sirakaya, M., &Cakmak, E. K. (2018). The effect of augmented reality use on achievement, misconception and course engagement. Contemporary Educational Technology. 9(3): 297-314. https://doi.org/10.30935/cet.444119 62 http://www.i-jim.org https://doi.org/10.17485/ijst/2015/v8is8/64711 https://doi.org/10.17485/ijst/2015/v8is8/64711 http://www.i-jim.org/ http://www.i-jim.org/ https://doi.org/10.3991/ijim.v14i14.12943 https://doi.org/10.4324/9781003117452 https://doi.org/10.4324/9781003117452 https://doi.org/10.1002/bmb.21341 https://doi.org/10.1002/bmb.21341 https://doi.org/10.1063/1.5055521 https://doi.org/10.1016/j.ieri.2012.06.174 https://doi.org/10.2147/vhrm.s46194 https://doi.org/10.2147/vhrm.s46194 http://www.i-jim.org/ http://www.i-jim.org/ https://doi.org/10.3991/ijim.v13i10.11305 https://doi.org/10.3991/ijim.v13i10.11305 https://doi.org/10.1177/0885066611404876 https://doi.org/10.11113/jt.v76.5843 https://doi.org/10.1080/00219266.2003.9655854 https://doi.org/10.30935/cet.444119 Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… [63] Soto, J. B., Ocampo, D. T., Colon, L. B., & Oropesa, A. V. (2020). Perceptions of ImmerseMe Virtual Reality Platform to Improve English Communicative Skills in Higher Education. International Journal of Interactive Mobile Technologies. 14(7): 4-19. https://doi.org/10.3991/ijim.v14i07.12181 [64] Sugiyono P. (2010). Metode Penelitian Kuantitatif, Kualitatif dan R&D (Qualitative and Quantitative Research Methods), Alfabeta. [65] Tait, A. R., Voepel-Lewis, T., & Levine, R. (2015). Using digital multimedia to improve parents’ and children's understanding of clinical trials. Archives of disease in childhood. 100(6): 589-593. https://doi.org/10.1136/archdischild-2014-308021 [66] Van der Heijden, H. (2003). Factors influencing the usage of websites: the case of a generic portal in The Netherlands. Information & Management. 40(6): 541-549. https://doi.org/10.1016/s0378-7206(02)00079-4 [67] Venkatesh, V. (2000). Determinants of perceived ease of use: integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information Systems Research. 11(4): 342-365. https://doi.org/10.1287/isre.11.4.342.11872 [68] Violante, M. G., &Vezzetti, E. (2015). Virtual interactive e‐learning application: An evaluation of the student satisfaction. Computer Applications in Engineering Education. 23(1): 72-91. https://doi.org/10.1002/cae.21580 [69] Woo, J. C. (2014). Digital game-based learning supports student motivation, cognitive success, and performance outcomes. Journal of Educational Technology & Society. 17(3): 291-307. [70] Zulkifli, A. N., Md Yussop, Y., Briki, W. A. M., Low, M. L., Chan, S. S., &Qasim, M. M. (2018). The hazards of smoking mobile app. Journal of Advanced Research in Dynamical and Control Systems. 10(10S): 1373-1378. [71] Zulkifli, A. N., MohdPozi, N. S., & Ismail, S. (2020). Young women’s perceptions towards the Hijab Try-on app. International Journal of Advanced Science and Technology. 29(6s): 1783-1795. 7 Authors Abdul Nasir Zulkifli is an Associate Professor at the School of Creative Industry Management & Performing Arts, Universiti Utara Malaysia. He has been teaching in various fields including Mechanical Engineering, Manufacturing Engineering, CAD, CAM, Management of Technology, Multimedia, Virtual Reality and 3D Animation for the last 34 years. His research areas include Virtual Reality, Augmented Reality, and Mobile applications in training and education. Email: abdulzulir@yandex.com Nur Fadziana Faisal Mohamed is currently a senior lecturer at the School of Multimedia Technology and Communication, Universiti Utara Malaysia. She has won gold medals for innovative products in numerous exhibitions at national and international levels. Her research interests include health behaviour change support systems, virtual / augmented reality, interaction design development and health informatics. Email: fadziana@uum.edu.my Mustafa Moosa Qasim is a lecturer in the Computer Science Department at the University of Basrah, Iraq. He has a bachelor degree in Computer Science, a master’s degree in Information Technology and Ph.D. in Information Technology. His research focus is in the field of Software Engineering, specializing in software architecture. He iJIM ‒ Vol. 15, No. 06, 2021 63 http://www.i-jim.org/ https://doi.org/10.3991/ijim.v14i07.12181 https://doi.org/10.1136/archdischild-2014-308021 https://doi.org/10.1016/s0378-7206(02)00079-4 https://doi.org/10.1287/isre.11.4.342.11872 https://doi.org/10.1002/cae.21580 mailto:abdulzulir@yandex.com fadziana@uum.edu.my Paper—Road Safety Education Courseware: A Study of Satisfaction and Learning Performance… finds himself consistently tend to the theories that investigate the behaviour, persuasion, motivation, and education, within the construct of mobile architecture and mobile application design. He has won several awards and medals through research grants. He is also considered as Ambassador in Journal Recent Patents on Computer Science from 2019 to present. His recent publications could be reached at SCOPUS ID, ORCID, Google Scholar, Researchgate, Academia, Publons. Email: mustafa.mq87@uobasrah.edu.iq Nur Afiqah Abu Bakar is an Administrative Officer at the Universiti Utara Malaysia Kuala Lumpur Campus. She has a bachelor degree and master’s degree in Information Technology specializing in Multimedia. Her research areas include Multimedia, Virtual Reality and Augmented Reality.Email: nurafiqah@uum.edu.my Article submitted 2020-12-19. Resubmitted 2021-01-27. Final acceptance 2021-01-29. Final version published as submitted by the authors 64 http://www.i-jim.org mailto:mustafa.mq87@uobasrah.edu.iq nurafiqah@uum.edu.my