International Journal of Interactive Mobile Technologies(iJIM) – eISSN: 1865-7923 – Vol 16 No 12 (2022) Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… Evaluation of Mobile Augmented Reality Hearing- Impaired Museum Visitors Engagement Instrument https://doi.org/10.3991/ijim.v16i12.30513 Esraa Jaffar Baker1(), Juliana Aida Abu Bakar2, Abdul Nasir Zulkifli2 1Computer Sciences Department, College of Science, Mustansiriyah University, Baghdad, Iraq 2School of Creative Industry Management and Performing Arts, Universiti Utara Malaysia, Changlun, Malaysia es-alshaibany@uomustansiriyah.edu.iq Abstract—Mobile Augmented Reality has come a long way since its inception as a multimedia system during the last few decades. From the concept of an Aug- mented Reality experience to actual smartphone applications, it has come a long way. Researchers have chosen to leverage the concept of engagement in the con- struction of Mobile Augmented Reality applications in order to boost museum visitors’ engagement and provide a more effective learning environment. The majority of Mobile Augmented Reality applications, on the other hand, were designed with normal hearing visitors in mind, while hearing-impaired visitors are given less attention. Those with hearing difficulties have an unfavourable experience as a result of this, and are unsatisfied with their visit. The elements of Mobile Augmented Reality aimed at engaging hearing-impaired museum visitors are determined in this study. The next step was to construct a conceptual model, which was then validated through an expert review. In investigating any flaws of the instrument among hearing-impaired museum visitors, a pilot study was con- ducted to improve the items and determine their level of reliability. Participants in the study were hearing-impaired who visited one of Iraq’s ancient museums. This study will focus on evaluating the prototype to see how effective it is at engaging hearing-impaired museum visitors in the near future. Keywords—mobile augmented reality, hearing-impaired, museum visitors, engagement 1 Introduction Augmented Reality (AR) combines computer-generated features with the real-world surroundings in creating a live environment, directly or indirectly [1]. It is established on mediated reality consisting of graphics, video, sound, and Global Positioning Sys- tem. It has been used in a number of fields, including advertisement [2], education [3], engineering [4], edutainment [5], manufacturing [6], and medicine [7]. Technologies and support devices have been produced from the applications which can improve con- sumers ’perceptions of reality and make life better for them. 114 http://www.i-jim.org https://doi.org/10.3991/ijim.v16i12.30513 mailto:es-alshaibany@uomustansiriyah.edu.iq Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… Mobile Augmented Reality (MAR) researchers are increasingly interested in improving museum visitors’ engagement, learning, enjoyment, and personalised expe- riences. This has been demonstrated in earlier research on interactive museum MAR applications, such as [8]; [9]; [10]; [11]; [12] and [13]. According to the findings of the aforementioned studies, the museum MAR application can provide learnable guid- ance to visitors while they are visiting the museum. However, according to [14], [15], and [16], most MAR applications for museum failed to engage users properly. User engagement is vital in museum visit since it improves learning, enjoyment, and accep- tance [16], [17], and [18]. As there are few research on the engagement of hearing-im- paired (HI) during museum visits, particularly employing MAR, this study focuses on HI engagement during museum visits. In assisting HI visitors, engagement elements must be identified. Studies by [19] and [20] have identified the elements of engagement which are subsequently utilised in proposing the conceptual model of Mobile Aug- mented Reality for Hearing-Impaired Museum Visitors Engagement (MARHIME). The identified engagement elements will be used to develop AR-enabled application to help individuals with hearing problems. Similar to the MAR, these elements may improve the HI user experience during museum visits. In this study, the MARHIME instrument was evaluated by experts in various fields. The Cronbach’s alphas for all of the elements were also utilised to interpret the instrument’s reliability in the pilot study. The purpose for these steps is to evaluate the instrument of this study and identify the limitations of the research instrument. 2 Marhime conceptual model As stated in a previous work [20], the MARHIME conceptual model is made up of six elements that were selected by experts. Engagement is represented by Aesthetics, Interaction, Usability, Motivation, Satisfaction, and Enjoyment in the model. Descrip- tions of the six elements are listed below. 2.1 Aesthetics Aesthetics is the study of natural beauty, visual appeal, and appealing mobile set- tings. This means that the MAR’s beauty must be noticeable in order for mobile users to be drawn to the application and the message representation provided by the MAR application. The element of Aesthetics was selected with three items which were adapted from the existing literatures [21]; [22] and [23]. 2.2 Interaction The state of being in command of an application is referred to as interaction whereby information, feedback, and interaction are provided in response to a specific action. This implies that a user and an application have a social relationship and are connected. The Interaction element was selected with three items which were adapted from the existing literatures [24] and [25]. iJIM ‒ Vol. 16, No. 12, 2022 115 Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… 2.3 Usability Usability refers to the application’s ease of use for the user and consistency of information. Ease of use is a tool for the evaluation of MAR applications and it helps user engagement with applications [26] and [24]. Usability was selected with three items which were adapted from the existing literatures [24] and [27]. 2.4 Motivation Motivation is the act of persuading a user to perform a specific action or activity. This means that motivation is a user’s ability to execute a task. Motivation is the desire to be excited about an application in order to meet a goal. The element Motivation was selected with three items which were adapted from the existing literatures [28] and [29]. 2.5 Satisfaction The act of being satisfied and enthusiastic about an application is referred to as satis- faction. It also applies to pleasant experiences with an application that lead to the user fulfilling the application’s expectations. The element Satisfaction was selected with three items which were adapted from the existing literatures [30]; [31] and [32]. 2.6 Enjoyment The sensation of pleasure, fun, and entertainment obtained through the use of an application is referred to as enjoyment. It also relates to users’ perceptions of how much the interactive application’s message has helped them. The element Enjoyment was selected with four items which were adapted from the existing literatures [33] and [34]. 3 Validity The degree to which a score accurately reflects a concept is defined as validity [35]. To put it differently, validity refers to a study’s use of an instrument that accurately assesses what it was designed to measure [36]. This paper discusses the content validity in the next section. 4 Content validity of an instrument Content validity refers to the extent to which the items’ content represents the suit- able universe of all relevant items under investigation [37]. It can be verified by content experts, instrument development experts (who are familiar with statistics and instru- ments) [38], and professional experts [37]. According to [39], there should be at least two academic experts. The minimum number for content validity, according to [40], is five. The content validity of the current study was determined by five academic experts. 116 http://www.i-jim.org Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… Meanwhile, at least one expert should be present for instrument construction, according to [39]. This was accomplished in this study, which included the appoint- ment of an expert from Universiti Utara Malaysia’s School of Quantitative Science. Professional experts consisted of HI where feedbacks from them were incorporated in the instrument’s items. 5 Expert review for the instrument The MARHIME instrument was reviewed by five experts from diverse institutions, and their fields of expertise included HI, MAR, AR, Museum, and Human-Computer Interaction (HCI). They are both male and female PhD holders with over five years of professional experience in their respective professions. A three-point scale comprising of D, M and R was used for the elements and items of the instrument [41]; [34]; [42] and [43]. The instruments were distributed using two methods: email and hand deliv- ery, since the experts were selected from various continents. All instruments were col- lected from the experts using the same method in which they were delivered to them. All of the experts responded, and some even made recommendations. Table 1 shows the responses from the experts. Table 1. Responses from expert review Element Items Definitely not Relevant (D) Relevant (R) Maybe not Relevant (M) Aesthetics A1 0 5 0 A2 0 5 0 A3 0 5 0 Usability U1 0 5 0 U2 0 4 1 U3 0 5 0 Interaction I1 0 5 0 I2 0 5 0 I3 0 4 1 Motivation M1 0 4 1 M2 0 5 0 M4 0 5 0 Satisfaction S1 0 5 0 S2 0 4 1 S3 0 5 0 Enjoyment E1 0 5 0 E2 0 5 0 E3 0 5 0 E4 0 4 1 iJIM ‒ Vol. 16, No. 12, 2022 117 Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… Dealing with HI people was tough [44]; [45]; [46], because of issues including low self-esteem, anxiety, and sadness, and they are often segregated from people who can hear normally [47]; [48]; [49]. They also have language and communication issues with normal people [50]; [44], therefore they pay little or no attention to issues [51]. As a result, HI people struggle to finish lengthy questionnaires [52]. According to [49], compared to normal people, HI are up to four times slower in terms of reading. As a consequence, in order to gather honest and realistic information and answers from them, surveys should include brief language and words that are simple and easy to read. The items listed in Table 2 are appropriate for the HI participants, based on these arguments. Table 2 shows the six MAR elements and all the 19 items. Since the 19-item scale was chosen by experts, the items’ content validity as well as the scale’s overall validity were both confirmed. 6 Data coding The initial stage in preparing data for analysis is coding of all the items and elements [36] as shown in Table 2. 7 Pilot study for marhime prototype It is a scaled-down version of a larger study intended to produce data for developing the scale and assessing its level of reliability [53]. A pilot study should be conducted to determine the suitability of those items because the current study has adapted items from many sources. Aside from ensuring item clarity, pilot study also demonstrates item layout and proper formulation depending on the difficulty or satisfaction of the participants when answering the questionnaire [55] and [56]. Cronbach’s alpha was employed since it determines whether an item measures the same thing as the one for which it was intended [57]. A total of 16 HI visitors were chosen to participate in the pilot study. According to [58], a sample size of 12 to 25 participants is required to obtain information concerning the pilot study’s flaws. To assure the accuracy of their responses, the participants were chosen from among those with the highest degree of education. All of the participants were given ques- tionnaires. As a result, some ambiguous wordings found during the pilot study were changed in the real evaluation to improve HI comprehension. This study proceeded to evaluate the MARHIME prototype among 16 HI museum visitors pertaining to their engagement. A pilot study was conducted before the HI museum visitors’ evaluation in order to achieve good findings from the responses of the participants. Data analysis was conducted to see if the MARHIME prototype succeeded in engaging the HI visitors. Future work includes the MARHIME prototype evaluation among HI Iraqi museum visitors. The HI participants in the pilot study session are depicted in Figure 1. 118 http://www.i-jim.org Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… Table 2. Elements and items for the marhime instrument Element Item Code Aesthetics Aesthetics item 1 A1 Aesthetics item 2 A2 Aesthetics item 3 A3 Usability Usability item 1 U1 Usability item 2 U2 Usability item 3 U3 Interaction Interaction item 1 I1 Interaction item 2 I2 Interaction item 3 I3 Motivation Motivation item 1 M1 Motivation item 2 M2 Motivation item 3 M4 Satisfaction Satisfaction item 1 S1 Satisfaction item 2 S2 Satisfaction item 3 S3 Enjoyment Enjoyment item 1 E1 Enjoyment item 2 E2 Enjoyment item 3 E3 Enjoyment item 4 E4 Fig. 1. Hi participants during pilot study 8 Results The pilot study saves time during the actual evaluation. In addition, the partici- pants’ responses were beneficial in improving some of the items because there were iJIM ‒ Vol. 16, No. 12, 2022 119 Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… weaknesses in formulating some of them due to the use of inappropriate words that might lead to misunderstanding [53]. Many researchers, including [59] and [60], advocated Cronbach’s alpha testing for the pilot study to ensure the reliability of the elements. This tool was used to see if the items measured the same thing that was intended [57]. The level of reliability, according to [54], spans from 0 to 1, with a minimum acceptability of 0.60 to 0.70. In addition, item analysis approach was used to show the most connected items to the construct using the Corrected Item-Total Correla- tion test, with any items with a value of less than 0.30 being eliminated [61]. A sample size of 12 to 25 participants, according to [54], is adequate to yield all of the relevant information on the flaws in the pilot study. 16 questionnaires were dis- tributed to HI visitors to verify the accuracy of their responses. As a result, certain ambiguous words were modified to improve the understanding in the actual evaluation. As shown in Table 3 and Figure 2, the Corrected Item-Total Correlation was higher than 0.30 for all of the items, ranging from 0.438 to 0.916. If Item Deleted, Cronbach’s Alpha ranges from 0.952 to 0.961, indicating that the items are highly reliable. Further- more, all constructs have achieved acceptable reliability, as Cronbach’s alpha ranges from 0.750 to 0.942, which is greater than 0.70 as suggested by [62]. 9 Conclusions The items for each element in this study were adapted from previous studies. They were validated by academic experts in the areas of HI, AR, MAR, HCI, and Museums. Items were selected by the experts based on suitability to the particular element. One of them even rectified several grammatical issues, and others offered suggestions for improving the items. In addition, before deploying the instrument on the HI users during the real evaluation, this study conducted a pilot study to confirm that the items were appropriate and to pre-test the instrument. The MARHIME conceptual model comprises of six MAR elements in engaging the HI museum visitors. All six elements and items were accepted by experts and also reliable through a pilot study. In the near future, this study will involve HI during a museum visit where an evaluation of the MARHIME prototype’s engagement capabilities will be undertaken whereby they will be given access to interact with the prototype. Table 3. Pilot study measurements’ and items’ reliability Elements Item Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted Cronbach’s Alpha Aesthetics Aesthetics item 1 0.771 0.956 0.757Aesthetics item 2 0.814 0.954 Aesthetics item 3 0.695 0.956 Usability Usability item 1 0.783 0.954 0.884Usability item 2 0.677 0.956 Usability item 3 0.778 0.954 (Continued) 120 http://www.i-jim.org Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… Elements Item Corrected Item-Total Correlation Cronbach's Alpha if Item Deleted Cronbach’s Alpha Interaction Motivation item 1 0.869 0.955 0.750Motivation item 2 0.553 0.957 Motivation item 3 0.752 0.955 Satisfaction Satisfaction item 1 0.902 0.952 0.942Satisfaction item 2 0.900 0.952 Satisfaction item 3 0.829 0.953 Enjoyment Enjoyment item 1 0.642 0.961 0.802 Enjoyment item 2 0.853 0.953 Enjoyment item 3 0.738 0.955 Enjoyment item 4 0.750 0.955 Fig. 2. Frequency measurement reliability of items 10 Acknowledgment This research was supported by Ministry of Higher Education (MoHE) of Malaysia through Fundamental Research Grant Scheme (FRGS/1/2019/WAB02/UUM/02/1). We also want to thank the School of Creative Industry Management and Performing Arts, and Universiti Utara Malaysia for the resources and support. Also, we would like to thank Mustansiriyah University (https://uomustansiriyah.edu.iq) Baghdad-Iraq for the support in the present work. Table 3. Pilot study measurements’ and items’ reliability (Continued) iJIM ‒ Vol. 16, No. 12, 2022 121 https://uomustansiriyah.edu.iq Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… 11 References [1] H. Arshad, W. K. Obeidy, & R. Z. Abidin, “An interactive application for halal products identification based on augmented reality”. International Journal on Advanced Science, Engineering and Information Technology, 7(1), 139–145, 2017. https://doi.org/10.18517/ ijaseit.7.1.1793 [2] S. J. Blodgett-Ford, W. Barfield, & A. Williams, “Advertising legal issues in virtual and augmented reality”. In Research Handbook on the Law of Virtual and Augmented Reality. Edward Elgar Publishing, 2018. https://doi.org/10.4337/9781786438591.00024 [3] M. H. A. Rahman, N. I. Utama, M. B. Ali, N. D. A. Halim, & S. Kasim, “The effect of aug- mented reality on spatial visualization ability of elementary school student”. International Journal on Advanced Science, Engineering and Information Technology, 9(2), 624–629, 2019. https://doi.org/10.18517/ijaseit.8.5.4971 [4] S. Baloch, S. Qadeer, & K. Memon, “Augmented Reality, a Tool to Enhance Conceptual Understanding for Engineering Students”, 2018. [5] B. Martínez, S. Casas, M. Vidal-González, L.Vera, & I. García-Pereira, “TinajAR: An Edutainment Augmented Reality Mirror for the Dissemination and Reinterpretation of Cultural Heritage”. Multimodal Technologies and Interaction, 2(2), 33, 2018. https://doi. org/10.3390/mti2020033 [6] V. Paelke, C. Röcker, & J. Bulk, “A Test Platform for the Evaluation of Augmented Real- ity Head Mounted Displays in Industrial Applications”. In International Conference on Applied Human Factors and Ergonomics (pp. 25–35). Springer, Cham, July, 2018. https:// doi.org/10.1007/978-3-319-94196-7_3 [7] B. W. Munzer, M. M. Khan, B. Shipman, & P. Mahajan, “Augmented reality in emergency medicine: a scoping review”. Journal of Medical Internet Research, 21(4), e12368, 2019. https://doi.org/10.2196/12368 [8] Z. He, L. Wu, & X. R. Li, “When art meets tech: The role of augmented reality in enhancing museum experiences and purchase intentions”. Tourism Management, 68, 127–139, 2018. https://doi.org/10.1016/j.tourman.2018.03.003 [9] M. C. Harrington, M. Tatzgern, T. Langer, & J. W. Wenzel, “Augmented reality brings the real world into natural history dioramas with data visualizations and bioacoustics at the carnegie museum of natural history”. Curator: The Museum Journal, 2019. https://doi. org/10.1111/cura.12308 [10] H. Jiang, X. L. Liu, X. Peng, M. X. Tang, D. He, H. L. Chen, K. Xiang, & B. Man, “3D Models to Educated Museum Interactive Exhibition with Computing Techniques”. In Com- puter Science On-line Conference (pp. 168–178). Springer, Cham, 2017. https://doi. org/10.1007/978-3-319-57141-6_18 [11] C. Scarles, M. Casey, & H. Treharne, “Enriching the visitor experience: Augmented reality and image recognition in tourism”. CAUTHE 2016: The Changing Landscape of Tourism and Hospitality: The Impact of Emerging Markets and Emerging Destinations, 1177, 2016. [12] M. Pérez-Sanagustín, D. Parra, R. Verdugo, G. García-Galleguillos, & M. Nussbaum, “Using QR codes to increase user engagement in museum-like spaces”. Computers in Human Behavior, 60, 73–85, 2016. https://doi.org/10.1016/j.chb.2016.02.012 [13] C. Y. Chen, B. R. Chang, & P. S. Huang, “Multimedia augmented reality information system for museum guidance”. Personal and Ubiquitous Computing, 18(2), 315–322, 2014. https:// doi.org/10.1007/s00779-013-0647-1 122 http://www.i-jim.org https://doi.org/10.18517/ijaseit.7.1.1793 https://doi.org/10.18517/ijaseit.7.1.1793 https://doi.org/10.4337/9781786438591.00024 https://doi.org/10.18517/ijaseit.8.5.4971 https://doi.org/10.3390/mti2020033 https://doi.org/10.3390/mti2020033 https://doi.org/10.1007/978-3-319-94196-7_3 https://doi.org/10.1007/978-3-319-94196-7_3 https://doi.org/10.2196/12368 https://doi.org/10.1016/j.tourman.2018.03.003 https://doi.org/10.1111/cura.12308 https://doi.org/10.1111/cura.12308 https://doi.org/10.1007/978-3-319-57141-6_18 https://doi.org/10.1007/978-3-319-57141-6_18 https://doi.org/10.1016/j.chb.2016.02.012 https://doi.org/10.1007/s00779-013-0647-1 https://doi.org/10.1007/s00779-013-0647-1 Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… [14] Y. L. Chang, H. T. Hou, C. Y. Pan, Y. T. Sung, & K. E. Chang, “Apply an augmented reality in a mobile guidance to increase sense of place for heritage places”. Educational Technology & Society, 18(2), 166–178, 2015. [15] C. Pollalis, W. Fahnbulleh, J. Tynes, & O. Shaer, “HoloMuse: Enhancing engagement with archaeological artifacts through gesture-based interaction with holograms”. In Proceedings of the Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 565–570). ACM, March, 2017. https://doi.org/10.1145/3024969.3025094 [16] C. Pollalis, A. Gilvin, L. Westendorf, L. Futami, B. Virgilio, D. Hsiao, & O. Shaer, “ARtLens: Enhancing Museum Visitors’ Engagement with African Art”. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 195–200). ACM, May 2018. https://doi.org/10.1145/3197391.3205435 [17] M. C. tom Dieck, T. H. Jung, & P. A. Rauschnabel, “Determining visitor engagement through augmented reality at science festivals: An experience economy perspective”. Computers in Human Behavior, 82, 44–53, 2018. https://doi.org/10.1016/j.chb.2017.12.043 [18] M. Hatala, & R. Wakkary, “Ontology-based user modeling in an augmented audio reality system for museums”. User Modeling and User-Adapted Interaction, 15(3–4), 339–380, 2005. https://doi.org/10.1007/s11257-005-2304-5 [19] E. J. Baker, J. A. Bakar, A., & A. N. Zulkifli, “A conceptual model of mobile augmented reality for hearing impaired museum visitors’ engagement”. International Journal of Interac- tive Mobile Technologies, 14(17), 79–96, 2020. https://doi.org/10.3991/ijim.v14i17.16649 [20] E. J. Baker, J. A. Bakar, A., & A. N. Zulkifli, “Elements of engagement in promoting social acceptance of mobile augmented reality application”. International Journal of Interactive Mobile Technologies, 14(17), 66–78, 2020. https://doi.org/10.3991/ijim.v14i17.16555 [21] E. N. Wiebe, A. Lamb, M., Hardy, & D. Sharek, “Measuring engagement in video game- based environments: Investigation of the User Engagement Scale”. Computers in Human Behavior, 32, 123–132, 2014. https://doi.org/10.1016/j.chb.2013.12.001 [22] H. L. O’Brien, “Antecedents and learning outcomes of online news engagement”. Journal of the Association for Information Science and Technology, 2017. https://doi.org/10.1002/ asi.23854 [23] H. L. O’Brien & E. G. Toms, “The development and evaluation of a survey to measure user engagement”. Journal of the American Society for Information Science and Technology, 61(1), 50–69, 2010. https://doi.org/10.1002/asi.21229 [24] M. K. Othman, H., Petrie, & C. Power, “Engaging visitors in museums with technology: Scales for the measurement of visitor and multimedia guide experience”. In IFIP Confer- ence on Human-Computer Interaction (pp. 92–99). Springer, Berlin, Heidelberg, September, 2011. https://doi.org/10.1007/978-3-642-23768-3_8 [25] M. H. Huang, “Designing website attributes to induce experiential encounters”. Computers in Human Behavior, 19(4), 425–442. 2003. https://doi.org/10.1016/S0747-5632(02)00080-8 [26] J. Sauro, “Measuring usability with the system usability scale (SUS)”. 2011. Verfügbar unter: http://www.measuringu.com/sus.php [22.04. 2013], 2015. [27] A. Hussain, H. I. Abubakar & N. B. Hashim, “Evaluating mobile banking applica- tion: Usability dimensions and measurements”. In Information Technology and Mul- timedia (ICIMU), 2014 International Conference on (pp. 136–140). IEEE, November, 2014. https://doi.org/10.1109/ICIMU.2014.7066618 [28] P. M. Chapman, “Models of engagement: Intrinsically motivated interaction with multime- dia learning software” (Doctoral dissertation, University of Waterloo), 1997. [29] B. J. Fogg, “A behavior model for persuasive design”. In Proceedings of the 4th international Conference on Persuasive Technology (p. 40). ACM, (2009, April). https://doi.org/10.1145/1541948.1541999 iJIM ‒ Vol. 16, No. 12, 2022 123 https://doi.org/10.1145/3024969.3025094 https://doi.org/10.1145/3197391.3205435 https://doi.org/10.1016/j.chb.2017.12.043 https://doi.org/10.1007/s11257-005-2304-5 https://doi.org/10.3991/ijim.v14i17.16649 https://doi.org/10.3991/ijim.v14i17.16555 https://doi.org/10.1016/j.chb.2013.12.001 https://doi.org/10.1002/asi.23854 https://doi.org/10.1002/asi.23854 https://doi.org/10.1002/asi.21229 https://doi.org/10.1007/978-3-642-23768-3_8 https://doi.org/10.1016/S0747-5632(02)00080-8 http://www.measuringu.com/sus.php https://doi.org/10.1109/ICIMU.2014.7066618 https://doi.org/10.1145/1541948.1541999 Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… [30] M. Alqahtani & H. Mohammad, “Mobile applications’ impact on student performance and satisfaction”. TOJET: The Turkish Online Journal of Educational Technology, 14(4), 2015. [31] J. P. Chin, V. A. Diehl & K. L. Norman, “Development of an instrument measuring user satisfaction of the human-computer interface”. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 213–218). ACM, May 1988. https://doi. org/10.1145/57167.57203 [32] S. F. Abdinnour-Helm, B. S. Chaparro, & S. M. Farmer, “Using the end-user computing satisfaction (EUCS) instrument to measure satisfaction with a web site”. Decision Sciences, 36(2), 341–364, 2005. https://doi.org/10.1111/j.1540-5414.2005.00076.x [33] M. MäNtymäKi & J. Salo, “Teenagers in social virtual worlds: Continuous use and pur- chasing behavior in Habbo hotel”. Computers in Human Behavior, 27(6), 2088–2097, 2011. https://doi.org/10.1016/j.chb.2011.06.003 [34] U. C. Pendit, S. B. Zaibon, & J. A. Abu Bakar. “Enjoyable informal learning at cultural heritage site using mobile augmented reality: Measurement and evaluation”. Journal of Tele- communication, Electronic and Computer Engineering (JTEC), 8(10), 13–21, 2016. [35] W. G. Zikmund, W. G. Babin, Barry, J. C. Carr, & M. Griffin, Business research methods (9th ed.): Cengage Learning, 2013. [36] U. Sekaran & R. Bougie, “Research methods for business: A skill building approach” (7th ed.): John Wiley & Sons, 2016. [37] D. R. Cooper & P. S. Schindler, “Business research methods” (12th ed.): McGraw-hill edu- cation, 2014. [38] L. L. Davis, “Instrument review: Getting the most from a panel of experts”. Applied Nursing Research, 5(4), 194–197, 1992. https://doi.org/10.1016/S0897-1897(05)80008-4 [39] C. F. Waltz, O. L. Strickland, & E. R. Lenz, “Measurement in nursing and health research”, (4th ed.): New York: Springer Publishing Company, 2010. [40] M. R. Lynn, “Determination and quantification of content validity”. Nursing Research, 35(6), 382–386, 1986. https://doi.org/10.1097/00006199-198611000-00017 [41] S. M. Sarif, N. Ibrahim, & N. Shiratuddin, “Design model of computerized personal deci- sion aid for youth: An expert review”. In AIP Conference Proceedings (Vol. 1761, No. 1, p. 020097). AIP Publishing, August, 2016. https://doi.org/10.1063/1.4960937 [42] R. Mason, B. McInnis, & S. Dalal, “Machine learning for the automatic identification of terrorist incidents in worldwide news media”. In Intelligence and Security Informatics (ISI), 2012 IEEE International Conference on (pp. 84–89). IEEE, June, 2012. https://doi. org/10.1109/ISI.2012.6284096 [43] N. Aziz, A. A. Mutalib, & S. M. Sarif, “Conceptual design model of assistive courseware for low vision (AC4LV) learners”. In International Conference on Advances in Educational Technology (ICAET’14) (pp. 1–12), 2014. https://doi.org/10.2991/icaet-14.2014.10 [44] A. Mishra, A. N. Nagarkar, & N. M. Nagarkar, “Challenges in education and employment for hearing impaired in India”. Journal of Disability Management and Special Education, 1(1), 35, 2018. [45] A. Abdul Mutalib, A. Salam, Nur, S. M. Mahmuddin, M. Ahmad, & S. N. Syed Yahya, “A concept of assertive courseware for hearing impaired learners”. ARPN Journal of Engineer- ing and Applied Sciences, 2015. [46] Y. T. Chen, “A study to explore the effects of self-regulated learning environment for hearing-impaired students”. Journal of Computer Assisted Learning, 30(2), 97–109, 2014. https://doi.org/10.1111/jcal.12023 [47] G. Batten, P. M. Oakes, & T. Alexander, “Factors associated with social interactions between deaf children and their hearing peers: A systematic literature review”. Journal of Deaf Stud- ies and Deaf Education, 19(3), 285–302, 2013. https://doi.org/10.1093/deafed/ent052 124 http://www.i-jim.org https://doi.org/10.1145/57167.57203 https://doi.org/10.1145/57167.57203 https://doi.org/10.1111/j.1540-5414.2005.00076.x https://doi.org/10.1016/j.chb.2011.06.003 https://doi.org/10.1016/S0897-1897(05)80008-4 https://doi.org/10.1097/00006199-198611000-00017 https://doi.org/10.1063/1.4960937 https://doi.org/10.1109/ISI.2012.6284096 https://doi.org/10.1109/ISI.2012.6284096 https://doi.org/10.2991/icaet-14.2014.10 https://doi.org/10.1111/jcal.12023 https://doi.org/10.1093/deafed/ent052 Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… [48] Lesar & H. S. Vitulic, “Personality traits of deaf and hard of hearing students from regular and special schools in Slovenia”. Solsko Polje, 24(3/4), 97, 2013. [49] N. K. Chuan, A. Sivaji, F. A. Loo, W. F. W. Ahmad, & S. S. Nathan, “Evaluating ‘Gesture Interaction’ requirements of mobile applications for deaf users: Discovering the needs of the hearing-impiared in using touchscreen gestures”. In Open Systems (ICOS), 2017 IEEE Con- ference on (pp. 90–95). IEEE, November 2017. https://doi.org/10.1109/ICOS.2017.8280280 [50] D. H. Barker, A. L. Quittner, N. E. Fink, L. S. Eisenberg, E. A. Tobey, J. K. Niparko, & CDaCI Investigative Team. “Predicting behavior problems in deaf and hearing children: The influences of language, attention, and parent–child communication”. Development and Psychopathology, 21(2), 373–392, 2009. https://doi.org/10.1017/S0954579409000212 [51] M. Bhuvaneswari & S. Immanuel, “Psychological Issues Among Hearing Impaired Adoles- cents”. Education Sciences & Psychology, 24(2), 2013. [52] N. M. M. Zainuddin, H. B. Zaman, & A. Ahmad, “Learning science using AR-book by blended learning strategies: A case study on preferred visual needs of deaf students”. Journal of Educational Technology Development and Exchange, 9(2), 5–20, 2009. [53] K. S. Bordens & B. B. Abbott, “Research design and methods: A process approach”, (8th ed.): McGraw-Hill, 2011. [54] J. F. Hair, M. Celsi, A. Money, P. Samouel, & M. Page, “The Essentials of Business Research Methods”, (3rd ed.): Taylor & Francis, 2016. https://doi.org/10.4324/9781315704562 [55] J. Adams, H. T. Khan, R. Raeside, & D. I. White, Research methods for gradu- ate business and social science students: SAGE publications India, 2007. https://doi. org/10.4135/9788132108498 [56] J. Creswell, “Planning, Conducting, and Evaluating Quantitative and Qualitative Research. America”, 2012. [57] R. F. DeVellis, “Scale development: Theory and applications” (Vol. 26): Sage publications, 2016. [58] P. B. Sheatsley, “Questionnaire construction and item writing”. Handbook of Survey Research, 4(1), 195–230, 1983. https://doi.org/10.1016/B978-0-12-598226-9.50012-4 [59] E. G. Carmines & R.A. Zeller, Reliability and Validity Assessment: SAGE Publications, 1979. https://doi.org/10.4135/9781412985642 [60] J. P. Peter, “Reliability: A review of psychometric basics and recent marketing practices”. Journal of Marketing Research, 6–17, 1979. https://doi.org/10.1177/002224377901600102 [61] J. Nunnally, Psychometric methods. In: New York: McGraw-Hill, (1978). [62] M. Tomitsch & T. Grechenig, Design Implications for a Ubiquitous Ambient Sound Display for the Deaf. In Conference & Workshop on Assistive Technologies for People with Vision & Hearing Impairments Assistive Technology for All Ages (CVHI 2007), 2007. 12 Authors Esraa Jaffar Baker is a computer science lecturer at Mustansiriyah University in Iraq. Her principal area of research includes multimedia systems, augmented real- ity, virtual reality and multimedia steganography. Her recent publications are in AIP Publishing, International Journal of Innovative Technology and Exploring Engineer- ing and Journal of Telecommunication, Electronic and Computer Engineering (JTEC). Email: es-alshaibany@uomustansiriyah.edu.iq Juliana Aida Abu Bakar is an associate professor at the School of Creative Industry Management & Performing Arts at Universiti Utara Malaysia. Virtual reality, iJIM ‒ Vol. 16, No. 12, 2022 125 https://doi.org/10.1109/ICOS.2017.8280280 https://doi.org/10.1017/S0954579409000212 https://doi.org/10.4324/9781315704562 https://doi.org/10.4135/9788132108498 https://doi.org/10.4135/9788132108498 https://doi.org/10.1016/B978-0-12-598226-9.50012-4 https://doi.org/10.4135/9781412985642 https://doi.org/10.1177/002224377901600102 mailto:es-alshaibany@uomustansiriyah.edu.iq Paper—Evaluation of Mobile Augmented Reality Hearing-Impaired Museum Visitors Engagement… augmented reality, and virtual heritage design, development and evaluation are among her research interests. Email: liana@uum.edu.my Abdul Nasir Zulkifli is an Associate Professor at the School of Creative Indus- try Management & Performing Arts at Universiti Utara Malaysia. Virtual Reality, Augmented Reality, and Mobile applications in training and education are among his research interests. Email: nasirzul@uum.edu.my Article submitted 2022-03-01. Resubmitted 2022-04-07. Final acceptance 2022-04-08. Final version published as submitted by the authors. 126 http://www.i-jim.org mailto:liana@uum.edu.my mailto:nasirzul@uum.edu.my