Short Paper—Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation https://doi.org/10.3991/ijim.v11i5.6879 Mostafa Al-Emran!"!# Universiti Malaysia Pahang, Malaysia. Al Buraimi University College, Oman malemran@buc.edu.om Said A. Salloum University of Fujairah, UAE The British University in Dubai, UAE Abstract—Mobile learning (M-learning) is a new wave in the era of educa- tional technology that provides informal, personal, voluntary, and situated learning opportunities for both learners and educators. Mobile-based assessment is one of the emerging technologies that attract many scholars to investigate its effectiveness due to the wide spectrum of its features like portability, interactiv- ity, flexibility, and ubiquity. Based on the surveyed literature, we noticed that there is a lack of studying the students' attitudes towards the utilization of mo- bile technologies in the context of e-Evaluation. The present study attempts to investigate the students' attitudes towards the utilization of mobile technologies in the e-Evaluation system of instructors. The data was collected through a questionnaire survey from Al Buraimi University College (BUC) in Oman. The total number of participants is 354 students. Findings indicated that 99% of the students own a mobile phone or tablet. Moreover, results indicated a statistically significant difference among the students’ attitudes in terms of their gender where the differences were in favor of male students. In contrast, results revealed no statistically significant differences among the students’ attitudes in terms of their age, degree, and department. Furthermore, other implications and future work are also reported in the study. Keywords—Mobile Technology, e-Evaluation, Higher Education, Oman. 1 Introduction M-learning motivates both cooperative and individualized learning experiences and provides opportunities for those learners who are not comfortable and convenient with the formal type of learning styles. Although M-learning systems are well developed for students to acquire their education, researchers reported that it could be further enhanced to acquire better results if there is some kind of systematic approach that could put forward for these systems. Hence, it is of utmost importance to develop strategies that could help with mobile learning processes [1]. iJIM ‒ Vol. 11, No. 5, 2017 195 Short Paper—Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation The elevated utilization of the mobile phones and PDAs has made it essential to acquire details about the assessments done through mobile devices. In the last few years, an increased number of research studies were conducted to examine the utiliza- tion of mobile devices in several sectors; one of them is the higher education [2], [3], [1], [4], [6], [5]. M-learning has a great influence on the students as well as research- ers for developing a robust educational infrastructure. The incorporation of mobile gadgets in the educational processes derives a new learning style named m-learning [7]. M-learning supports various dimensions like mobility of technology, mobility of learners, mobility of educators, and mobility of learning [2]. Instructors are handing out their syllabus, course contents, materials, assignments, announcements, and assessments to their students via online platforms through the aid of the Internet [8]. The increased usage of mobile phones has created the need for Mobile learning a necessity in educational sector [9]. The progress of mobile and wireless communication technologies has stimulated an increasing number of studies concerning M-learning, in which students acquire learning without being restricted to time and place [1]. E-assessment makes it easy for the educators to mark the students' papers in an automatic manner and feed the students back with their results immedi- ately. The purpose of this paper is to investigate the students' attitudes towards the utilization of mobile technologies in the e-Evaluation system of instructors. The paper is organized as follows: section 2 gives a summarized history of using mobile devices in e-assessment in the higher education. Section 3 explains the re- search methodology. Section 4 shows the results of the study. Conclusion and future perspectives are described in section 5. 2 Literature Review Nowadays, Mobile technology is being used very progressively in many sectors; especially in education. The performed assessments through mobile devices are recent and have a lot of aspects in various dimensions. Time should be invested in acquiring more information about M-learning technology before it is being implemented for educational purposes. That is, researchers should examine how the users would react to the e-assessment using the M-learning technology. A study by [10] conducted a research that intended to solve the problems of assessment while using the smartphones for assessment. This is implemented by conducting a trial and survey to determine the prototype that consists of experienced specialists in assessments. The survey was based on "Ease of Use", "Satisfaction", "Value", and "length of assess- ment". The increased usage of smartphones has made necessity that mobile communi- cation equipment like mobile phones and PDAs are observed in details before being used for assessments. Several studies were conducted regarding the importance of assessment in the educational process [11], [12], [13], and [14]. PDAs were used for a competency-based assessment by the undergraduate medical students in their final year of study [15]. The study indicated that mobile assessment through the use of PDAs is very clear, simple, easy-to-use, and provides better feed- back. Hence, this allows the students to increase their learning abilities through the 196 http://www.i-jim.org Short Paper—Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation utilization of PDAs for learning purposes. A study by [16] conducted a research study about web-based personality assessment through computers and smart phones. Find- ings showed that there was a lot of variation in the time used for accomplishing the assessments through smartphones and computers. A study by [8] studied the students’ perceptions towards the results of the assessment and the influence on students' productivity through taking the mobile-based assessment. Results revealed that stu- dents tend to like the mobile-based assessment as it is user-friendly, simple, and get- ting quick results through the internet using their smartphones. Authors of [2] stated that 99% of the students in the Gulf region countries (Oman & UAE) own Smartphone/tablets. The study showed that many students were using their mobile devices in the learning-teaching process, surfing the web, and checking their emails. Additionally, the study indicates that students are highly inclined towards suing smartphones for educational purposes. By investigating the literature, we noticed that there is a lack of studying the stu- dents' attitudes towards the utilization of mobile technologies in the context of e- Assessment, generally, and more specifically in the Gulf region countries. A study by [17] claimed that studying the attitudes toward any technology assist the decision makers to identify the shortcomings and strengths and contribute to the establishment of a reliable infrastructure. Al Buraimi University College (BUC) is one of the evolv- ing colleges in Oman that is keen to provide reliable technologies to their staff and students; one of such technologies is the e-Evaluation system [18]. Accordingly, this creates a need for examining the students' attitudes towards the utilization of mobile technologies in the e-Evaluation system of instructors at BUC. Though, we are seek- ing to answer the following research questions: RQ1: Is there any significant difference among the students’ attitudes towards the use of mobile technologies for e-Evaluation in terms of gender? RQ2: Is there any significant difference among the students’ attitudes towards the use of mobile technologies for e-Evaluation in terms of age? RQ3: Is there any significant difference among the students’ attitudes towards the use of mobile technologies for e-Evaluation in terms of degree? RQ4: Is there any significant difference among the students’ attitudes towards the use of mobile technologies for e-Evaluation in terms of department? 3 Research Methodology 3.1 Sample and Study Instrument The data was collected by using a questionnaire survey. Surveys were conducted at Al Buraimi University College (BUC) in Oman. The total number of participants is 354 students. According to [2], we followed the "Purposive Sampling approach" in which the participated students were easily reachable and eager to participate in the study. Students from different departments, different majors, and various age groups were taken part in the study. Results indicated that 61.9% of the participants are fe- males while the others are males. Additionally, 72.3% of the participants’ ages are iJIM ‒ Vol. 11, No. 5, 2017 197 Short Paper—Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation between 18-22 years old. In terms of department, 49.7% of the participants are IT students. Regarding the degree, 55.6% of the participants are bachelor degree stu- dents. 3.2 Survey Structure A questionnaire survey was organized and distributed among the students. The stu- dents’ survey consists of two sections. The first section includes the demographic data of the participants in addition to their mobile technology usage. The second section consists of eight items that represent the students’ attitudes toward the use of mobile technology in the e-Evaluation system. A five-point Likert Scale with strongly agree (5), agree (4), neutral (3), disagree (2), and strongly disagree (1) has been used to measure the (8 items). 3.3 Measurement Analysis According to the recent studies by [3], [2], [19], a reliability test was computed for the (8 items) that characterize the students’ attitudes towards the utilization of mobile devices in e-Evaluation by calculating the Cronbach’s alpha. The research claimed that the reasonable threshold for Alpha values is 0.7 or higher. Our study showed that the Cronbach’s alpha value for the (8 items) is (Alpha = 0.925). This indicates that the variables are extremely reliable since the calculated alpha value is greater than 0.7. 4 Findings and Discussion In addition to the illustrated descriptive statistics in the research methodology sec- tion, our results indicated that 99% of the participants own a mobile phone or tablet. Our results are very consistent with a recent study by [2] that showed the same per- centage for the same purpose in the same region. These results confirmed that stu- dents in the Arab Gulf region countries are highly interested in the utilization of mo- bile technologies for performing their tasks. Moreover, our results revealed that 94% of the students were utilizing their mobile technologies for evaluating their instructors by the end of each semester as per BUC policies. The following shows the analysis for each research question. RQ1: Is there any significant difference among the students’ attitudes towards the use of mobile technologies for e-Evaluation in terms of gender? An independent sample t-test was performed for examining the existence of any statistically significant difference among the students’ attitudes towards the utilization of mobile technologies (smartphone/tablet) for evaluating their instructors with regard to their gender. According to Table 1, results indicated a statistically significant dif- ference among the students with regard to their gender (p = 0.033, p <= 0.05). The differences were in favor of male students. Nevertheless, other studies like [2] and [19] didn’t show any statistical differences in terms of gender. 198 http://www.i-jim.org Short Paper—Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation Table 1. Differences among the students’ attitudes with regard to gender. Gender N Mean Std. Dev. t df Sig Male 135 3.8898 0.88389 4.997 352 0.033 Female 219 3.3664 0.99957 RQ2: Is there any significant difference among the students’ attitudes towards the use of mobile technologies for e-Evaluation in terms of age? Means and standard deviations for the students’ age groups were computed for in- vestigating the existence of any significant difference among the students’ attitudes towards the utilization of mobile technologies (smartphone/tablet) for evaluating their instructors with regard to their age. Moreover, a one-way analysis of variance (ANO- VA) has been computed for determining whether if any statistically significant differ- ence exists among the mean values. According to Table 2, our results revealed that there were no statistically significant differences (p = 0.693, p > 0.05) among the students’ attitudes in terms of their age and the computed F-score is (0.485). These results could be attributed to the reason that all age groups are highly motivated for evaluating their instructors using their mobile technologies. Table 2. ANOVA results for students attitudes’ with regard to age. Sum of Squares df Mean Square F Sig. Between Groups 1.428 3 0.476 0.485 0.693 Within Groups 343.950 350 0.983 Total 345.378 353 RQ3: Is there any significant difference among the students’ attitudes towards the use of mobile technologies for e-Evaluation in terms of degree? Means and standard deviations for the students’ degrees were measured for inves- tigating the existence of any significant difference among the students’ attitudes to- wards the utilization of mobile technologies (smartphone/tablet) for evaluating their instructors with regard to their degree. Additionally, a one-way analysis of variance (ANOVA) has been computed for determining whether if any statistically significant difference exists among the mean values. According to Table 3, our results revealed that there were no statistically significant differences (p = 0.225, p > 0.05) among the students’ attitudes in terms of their degree and the computed F-score is (1.480). These results could be referred to the fact that all students are knowledgeable about the e- Evaluation system running at BUC and they are aware of using that system using their mobile technologies regardless of their degrees. Table 3. ANOVA results of students' attitudes with regard to degree. Sum of Squares df Mean Square F Sig. Between Groups 1.446 1 1.446 1.480 0.225 Within Groups 343.932 352 0.977 Total 345.378 353 iJIM ‒ Vol. 11, No. 5, 2017 199 Short Paper—Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation RQ4: Is there any significant difference among the students’ attitudes towards the use of mobile technologies for e-Evaluation in terms of department? Means and standard deviations for the students’ majors in terms of departments were measured for investigating the existence of any significant difference among the students’ attitudes towards the utilization of mobile technologies (smartphone/tablet) for evaluating their instructors with regard to their departments. Furthermore, a one- way analysis of variance (ANOVA) has been carried out for determining whether if any statistically significant difference exists among the mean values. According to Table 4, our results revealed that there were no statistically significant differences (p = 0.276, p > 0.05) among the students’ attitudes in terms of their department and the computed F-score is (1.295). Similarly, a recent study by [2] didn’t show any statisti- cal differences in terms of major. Table 4. ANOVA results of students' attitudes with regard to department. Sum of Squares df Mean Square F Sig. Between Groups 3.792 3 1.264 1.295 0.276 Within Groups 341.587 350 0.976 Total 345.378 353 5 Conclusion and Future Work Mobile-based assessment is one of the emerging technologies that attract many scholars to examine its effectiveness due to the wide spectrum of its features like portability, interactivity, flexibility, and ubiquity. Based on the surveyed literature, we noticed that there is a lack of studying the students' attitudes towards the utilization of mobile technologies in the context of e-Evaluation in the Gulf region countries. Al Buraimi University College (BUC) is one of the evolving colleges in Oman that is keen to provide reliable technologies to their staff and students; one of such technolo- gies is the e-Evaluation system. Consequently, this creates a need for investigating the students' attitudes towards the utilization of mobile technologies in the e-Evaluation system of instructors at BUC. The data was collected through the use of a question- naire survey from BUC. The total number of participants is 354 students. Findings indicated that 99% of the participants own a mobile phone or tablet. Results indicated a statistically significant difference among the students in terms of their gender where the differences were in favor of male students. On the other side, results revealed no statistically significant differences among the students’ atti- tudes in terms of their age, degree, and department. In accordance with the descriptive statistics, our results revealed that 94% of the students were using their mobile tech- nologies for evaluating their instructors by the end of each semester as per the college policies. Accordingly, the non-significant differences could be attributed to the reason that almost all of the students are highly knowledgeable about the e-Evaluation sys- tem running at BUC and they are aware of using that system using their mobile tech- nologies irrespective of their age, degree, and department. 200 http://www.i-jim.org Short Paper—Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation This study is limited to only four factors; namely gender, age, degree, and depart- ment. As a future work, we are interested in incorporating other factors like level of students (low, medium, and high), passed credit hours, and mobile technology owner- ship for better understanding these attitudes regarding the usage of mobile technolo- gies in the e-Evaluation system of educators. 6 References [1] Hwang, G. J., & Chang, H. F. (2011). A formative assessment-based mobile learning ap- proach to improving the learning attitudes and achievements of students. Computers & Education, 56(4), 1023-1031. https://doi.org/10.1016/j.compedu.2010.12.002 [2] Al-Emran, M., Elsherif, H. M., & Shaalan, K. (2016). Investigating attitudes towards the use of mobile learning in higher education. Computers in Human Behavior, 56, 93-102. https://doi.org/10.1016/j.chb.2015.11.033 [3] Al-Emran, M., & Shaalan, K. (2015a). Attitudes towards the use of mobile learning: a case study from the gulf region. International Journal of Interactive Mobile Technologies (iJIM), 9(3), 75-78. https://doi.org/10.3991/ijim.v9i3.4596 [4] Iqbal, S., Chowdhury, M. U., & Harsh, O. K. (2013, September). Mobile devices support- ed learning for novice programmers. In e-Learning and e-Technologies in Education (ICEEE), 2013 Second International Conference on (pp. 277-282). IEEE. https://doi.org/10.1109/ICeLeTE.2013.6644388 [5] Al-Emran, M., & Shaalan, K. (2017). Academics’ Awareness Towards Mobile Learning in Oman. Int. J. Com. Dig. Sys, 6(1). [6] Al-Emran, M. N. H. (2014). Investigating Students’ and Faculty members’ Attitudes To- wards the Use of Mobile Learning in Higher Educational Environments at the Gulf Re- gion (Doctoral dissertation, The British University in Dubai (BUiD)). [7] Nikou, S. A., & Economides, A. A. (2013). Student achievement in paper, computer/web and mobile based assessment. In BCI (Local) (p. 107). [8] Karadeniz, !. (2009). The impacts of paper, web and mobile based assessment on students achievement and perceptions. Scientific Research and Essays, 4(10), 984-991. [9] Miguel, J., Caballé, S., Xhafa, F., Prieto, J., & Barolli, L. (2016). A methodological ap- proach for trustworthiness assessment and prediction in mobile online collaborative learn- ing. Computer Standards & Interfaces, 44, 122-136. https://doi.org/10.1016/j.csi.2015. 04.008 [10] Wong, C. C., Sellan, R., & Lee, L. Y. (2006). Assessment using mobile phone—an explor- atory study. In Proceedings of IAEA 2006 Conference, Grand Copthorne Waterfront Ho- tel, Singapore City, Singapore. [11] Chen, Y. S., Kao, T. C., & Sheu, J. P. (2003). A mobile learning system for scaffolding bird watching learning. Journal of Computer Assisted Learning, 19(3), 347-359. https://doi.org/10.1046/j.0266-4909.2003.00036.x [12] Hsi, S. (2003). A study of user experiences mediated by nomadic web content in a muse- um. Journal of Computer Assisted Learning, 19(3), 308-319. https://doi.org/10.1046/j.02 66-4909.2003.jca_023.x [13] Seppälä, P., & Alamäki, H. (2003). Mobile learning in teacher training. Journal of com- puter assisted learning, 19(3), 330-335. https://doi.org/10.1046/j.0266-4909.2003.00034.x [14] Relan, A., Parker, N., Wali, S., Guiton, G., & Fung, C. C. (2004). Supporting handheld technologies in a medical school curriculum: lessons from three years of design, develop- ment and implementation. In Wireless and Mobile Technologies in Education, 2004. Pro- iJIM ‒ Vol. 11, No. 5, 2017 201 Short Paper—Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation ceedings. The 2nd IEEE International Workshop on (pp. 51-58). IEEE. https://doi.org/10.1109/WMTE.2004.1281333 [15] Coulby, C., Hennessey, S., Davies, N., & Fuller, R. (2011). The use of mobile technology for work!based assessment: the student experience. British Journal of Educational Tech- nology, 42(2), 251-265. https://doi.org/10.1111/j.1467-8535.2009.01022.x [16] Huff, K. C. (2015). The comparison of mobile devices to computers for web-based as- sessments. Computers in Human Behavior, 49, 208-212. https://doi.org/10.1016/j.chb.2015 .03.008 [17] Al-Emran, M., & Shaalan, K. (2015b, August). Learners and educators attitudes towards mobile learning in higher education: State of the art. In Advances in Computing, Communications and Informatics (ICACCI), 2015 International Conference on (pp. 907- 913). IEEE. [18] Al Emran, M., & Shaalan, K. (2014, March). E-podium Technology: A medium of manag- ing Knowledge at Al Buraimi University College via M-learning. In BCS International IT Conference. [19] Al-Emran, M., & Malik, S. I. (2016). The Impact of Google Apps at Work: Higher Educa- tional Perspective. International Journal of Interactive Mobile Technologies (iJIM), 10(4), 85-88. https://doi.org/10.3991/ijim.v10i4.6181 7 Authors Mostafa Al-Emran (corresponding author) is a PhD student in Computer Sci- ence. He has graduated from The British University in Dubai with a distinction level along with the top Academic Excellence Award with MSc in Informatics (Knowledge and Data Management). His main research interest includes: M-Learning, Knowledge Management, Educational Technology and Text Mining. Said A. Salloum is currently a master student of Informatics (Knowledge and Data Management) at The British University in Dubai. He is currently the Director of Computer Center at University of Fujairah. Salloum is an Oracle expert since 2013 along with various recognized international certificates that are issued by Oracle. Article submitted 12 March 2017. Published as resubmitted by the authors 26 April 2017. 202 http://www.i-jim.org iJIM – Vol. 11, No. 5, 2017 Students' Attitudes Towards the Use of Mobile Technologies in e-Evaluation