International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 06, 2021 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App An Instrumental Assessment of Touch‘n Go eWallet Mobile App https://doi.org/10.3991/ijim.v15i06.20605 Azham Hussain () Universiti Utara Malaysia, Sintok, Malaysia hussainazh1217@gmail.com Emmanuel O.C. Mkpojiogu Universiti Utara Malaysia, Sintok, Malaysia Veritas University, Abuja, Nigeria Fazillah Mohmad Kamal, Rohazna binti Wahab, Noor Halawati Che Meh Universiti Utara Malaysia, Sintok, Malaysia Abstract—This study assessed the instrumentality of Touch ’n Go eWallet mobile app at selected areas in University Malaysia Perlis (UniMAP) and Politeknik Tuanku Syed Sirajuddin (PTSS) Perlis, Malaysia, in July and August 2019. Fifteen staff from the two institutions was selected as participants. The pur- pose of the test was to assess the usability of the app and get user feedback to improve the instrumental quality of the application in order to meet user satisfac- tion and their experience. This report contains the participants’ feedbacks, task completion rates, ease or difficulty of task completion, time on task, errors, and recommendations for improvements. This study used 4 tasks to assess the instru- mental quality of Touch ‘n Go eWallet mobile app. Overall, the outcome of the study revealed that the app is generally usable and instrumental to assisting users accomplish their electronic wallet goals. There is however some observed issues in the app that require fixing to enhance the instrumental quality of app. Keywords: eWallet app, instrumental assessment, mobile application, touch ‘n go, usability test 1 Introduction Touch ‘n Go eWallet is an integrated mobile application. It uses a physical Touch ‘n Go Card. The app allows users to add in Physical Touch ‘n Go card number easier to track their transaction. Other functions available with the application to make daily use easier are as follows: i) Transfer money to other Touch ‘n Go eWallet users; ii) Reload mobile prepaid; iii) Pay for utilities and postpaid bills; iv) Purchase movie and flight tickets; v) Dynamic QR code payment at participating Touch ‘n Go eWallet merchants; and vi) Pay tolls via RFID feature and TNG Card feature. An instrumental assessment is the evaluation that ascertains how an application is instrumental to or successful in 4 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i06.20605 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App enabling users to achieve their goals in an interaction with or in their use of an applica- tion. Usability evaluation is one way this can be done. Usability testing for mobile ap- plications is highly important for both entrepreneurs and users. A well-tested software application helps to establish reliable and fruitful collaboration between business own- ers and prospective customers. Evaluative improvement of a user interface is crucial for achieving high usability and instrumentality quality [1-7]. In this study, the researchers conducted a usability assessment of the Touch ‘n Go eWallet mobile application using participants’ mobile phone. The mobile phones have in-built video camera to record the participants’ action, comments and navigation choices. The participants were free to choose any place for testing the mobile application. The usability testers were present in the selected areas. The session captured each participant’s navigational choices, task completion rate, observations, questions and feedbacks. Usability testing includes establishing a baseline for user performance, establishing and validating user performance measures, and identifying prospective design issues that need to be addressed to improve product effectiveness, efficiency, reliability, and productivity as well as the satisfaction of end-users. Usability testing offers the oppor- tunity of finding out how well the application goals are being achieved [7-15]. The aims of this study are inter alia: i) To evaluate the usability of Touch ‘N Go eWallet appli- cations; ii) To recommend improvements to the design of the Touch ‘N Go eWallet applications where necessary. This research will assist developers to identify the strength and weakness of the application. Implementation of the suggestions and rec- ommendation made will go a long way to improve the application and increase business reputation and customer engagement. 2 Methodology Sessions: Each participant was allowed to choose and decide where to conduct the testing/evaluation. This could be in their personal rooms that is conducive and quiet or in any other place. The participants’ rooms should have at least one chair and one table in order to make the participants more comfortable. In addition, pens were needed. All user interaction will be captured by using the smartphone video camera. This will rec- ord how users perform the tasks given. The participants were informed that it is the eWallet application that is being tested and not their performance per se as participants, and that the goal of the assessment was to report anything that impedes their experience of using eWallet during their interaction activities and the carrying out of the tasks. Participants were also told to think-aloud while carrying out their tasks, reporting any- thing that impacts or impedes their experience. They were also told to feel free to com- ment on things they like or do not like about the eWallet application, thereby encour- aging them to speak more. The test administrator recorded each session while watching the behavior of the participants. If any of the participant was not talking sufficiently, the test administrator will help such participant by asking him/her some brief questions, endeavoring to be as impartial as possible (e.g. by requesting "How do you find the link icon so far?" instead of "Do you discover the controls difficult?"). Thus, such questions are not to suggest answers to the particular participant. After the task sessions, the post- iJIM ‒ Vol. 15, No. 06, 2021 5 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App task questionnaire was administered and completed. Lastly, after task completion, users were asked to answer the questionnaire provided to collect their feedback. The measures collected are: i) Demographic Information, ii) Satisfaction Assessment (Ob- jective metrics & Subjective metrics) and iii) Recommendations. The allocated of roles and responsibility for each test session is as follows: Two members of the research team were saddled with the following responsibilities: i) Briefing participants on the scope of the usability testing; ii) Observe each participant in each test sessions; iii) Interview participants after their testing; and iv) Recording of the participants’ test sessions. On the other hand, the participants have the following responsibilities too: i) Test the func- tions in Touch ‘n Go eWallet Application; and ii) Complete a data sheet form [16-26]. Participants: The targeted user group is the Touch ‘n Go eWallet users. The test group consists of 15 persons (5 staff members from Tuanku Syed Sirajuddin Polytech- nic and 10 staff members from UniMAP) from various units. The test was conducted from July 2019 to August 2019. Below is the demographic information for the selected participants: of the 15 participants, 1 (6.70%) was male while the other 14 were female (63.30%). Two of the participants’ age ranged from 26 and below (13.30%), one par- ticipant fell into the 26-35 age bracket (6.70%). Most of the participants’ (10) age ranged from 36-45 (66.70%). Lastly, two participants were aged 46 and above (13.30%). Evaluation Tasks / Scenarios: The usability test for Touch ‘n Go eWallet Applica- tion tested several functions. These include: i) Check Account Balance; ii) Perform money transfers to a favorite eWallet user; iii) Reload eWallet (FPX Online Banking, Credit or Debit Card, Touch 'n Go eWallet Reload Pin, Auto-Reload); and iv) Perform payment at selected attached merchants. Fig. 1. Touch ‘n Go eWallet selected functions 6 http://www.i-jim.org Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App The participants were not facilitated on how to use the application. It is assumed that they have previous experience with smartphone applications and are not technology illiterate. The participant’s responsibilities are to complete a set of tasks given by the test facilitators and presented to them before the application is tested. They are to per- form the several tasks at selected Touch ‘n Go eWallet merchants to get their feedback regarding the application. The participants are free to test the application in their own way. Usability Metrics: Usability metrics refers to user performance measures measured against specific performance goals necessary to satisfy usability requirements. This ex- plains the instrumentality of the app (that is, how instrumental the app is to users in enabling them accomplish their tasks). In this study, task completion/success rates, ad- herence to dialog scripts, and error rates was used. Time-to-completion of tasks was collected [17-34]. Table 1 displays the metrics selected for this Touch ‘n Go eWallet application. Table 1. Objective and Subjective Metrics for the testing Objective Metrics Subjective Metrics • Time taken to complete the task • Satisfaction with application interface • Time taken for application to response • Satisfaction with the use of text • Time taken to key-in/process data • Satisfaction with the use of color • Number of Internet connection crashes • Satisfaction with the graphics and sound • Number of mistakes while key-in data • Satisfaction with the menu provided • Task is successful? • Satisfaction with the output • Satisfaction with the application performance • Easy to find help • Enjoyed the application? Testing Requirement: The hardware requirement for this test is a smartphone while the software requirements include the following: i) Touch ‘n Go eWallet App; ii) Screen Recording App; iii) Smart Phone Video Camera. Testing Analysis: In each session, the participants completed the tasks in a reason- able time and their actions were recorded. The testing proceeded even if the participant faced any failure. The failure was however recorded. The test tasks are as follows: Task 1: Check account balance-This task will demonstrate how the user will check the ac- count balance in the Touch ‘n Go eWallet; Task 2: Reload eWallet (FPX Online Bank- ing, Credit or Debit Card, Touch 'n Go eWallet Reload Pin, Auto-Reload) - This task will demonstrate how the user will reload their Touch ‘n Go eWallet account using selected reload option; Task 3: Perform money transfers - This task will demonstrate how the user will transfer money from their Touch ‘n Go eWallet account to a favorite eWallet user; Task 4: Perform payment at selected attached merchants - This task will demonstrate how the user will perform payment at selected attached merchants. Data Collection Instruments: There are several forms used in order to get feedback from the participants during the test: i) Data Collection Sheet: The data collection sheet is used to get feedback from the participants after all the tasks have been tested; ii) Consent Form: The Consent Form was created to obtain the consent of the users to iJIM ‒ Vol. 15, No. 06, 2021 7 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App participate in the study. The test procedure and their role as testers are clearly stated in the form. The testing activity is voluntary and no participant is coerced. Participants are informed that they can opt out of the study at any time they so wish. 3 Results Task Completion / Success Rate: All participants (100%) successfully completed Task 1 (check account balance). This is the simplest task that participant tested since the account balance displayed at the top of screen with a big font size with “RM” label right after the apps is opened. In addition, all participants (100%) were able to accom- plish Task 2 (reload money). This task requires participants to log in to their personal account to reload their eWallet in a secured way (mentioned in the apps). The account balance then updates right after the finishing of the reload process from a selected op- tion (FPX Online Banking, Credit or Debit Card, Touch 'n Go eWallet Reload Pin, Auto-Reload). Similarly, all participants (100%) also successfully completed the Task 3 (perform money transfer). A task completion message is delivered by the application. All participants managed to finish Task 4 as well (perform payment at selected attached merchants). Task 4 appeared to be a challenging one because participants were required to present themselves at the selected merchant, but the process to be completed is an easy one. The payment process just needed a click on the menu “Pay” then the QR code will be displayed and scanned by the merchant cashier. The account will then be de- ducted and the balance displayed on the app. Table 2 shows the participants’ task com- pletion. Table 2. Task Completion Success Rates Participant Task 1 Task 2 Task 3 Task 4 1 √ √ √ √ 2 √ √ √ √ 3 √ √ √ √ 4 √ √ √ √ 5 √ √ √ √ 6 √ √ √ √ 7 √ √ √ √ 8 √ √ √ √ 9 √ √ √ √ 10 √ √ √ √ 11 √ √ √ √ 12 √ √ √ √ 13 √ √ √ √ 14 √ √ √ √ 15 √ √ √ √ Total Success 15 15 15 15 Success Completion Rates 100% 100% 100% 100% 8 http://www.i-jim.org Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App Time on Task: The screen recording app recorded the time on task for each partici- pant. Some tasks were inherently more difficult to complete than others and this af- fected the average time on task. Task 1 required participants to check their account balance and it took the shortest time to complete (mean = 16.40 seconds). Completion times for task 1 ranged from 7 to 55 seconds with most participants completing their tasks in less than 20 seconds. There were two participants who took longer time to complete the first task with 28 and 55 seconds respectively. Task 2 required participants to reload money and the average time on task for this task was 150.13 seconds. Com- pletion times for task 2 ranged from 62 to 260 seconds with most participants accom- plishing their tasks in less than 200 seconds. There was however three participants who took more than 200 seconds to complete task 2. Task 3 required the participants to perform money transfer and this task took the longest time to complete (mean = 192 seconds). Howbeit completion times ranged from 37 to 535 seconds with most partici- pants completing their tasks in less than 200 seconds. This task is more complicated compared to the other tasks. It widened the range in task completion times. One partic- ipant however took almost 9 minutes (completion time =535 seconds) to complete task 3. Task 4 required the participants to perform payment at selected attached merchants and its mean completion time was 34.80 seconds. The completion times ranged from 17 to 60 seconds with most participants completing their tasks in not more than 60 seconds (less than 4 minutes). Table 3 shows the task completion times per participant. Table 3. Time on Task P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 Avg. TOT Task 1 28 20 19 15 19 14 7 55 10 8 9 19 7 8 8 16.40 Task 2 120 161 260 62 155 120 205 180 131 178 258 148 75 77 122 150.13 Task 3 230 245 272 40 535 300 188 300 280 200 37 59 49 53 92 192.00 Task 4 20 36 40 39 60 17 30 21 37 35 53 23 37 25 49 34.80 Task Errors: In task 2, four (4) out of the 15 participants made typing error mistakes but it is in minimum errors range of 1-3 typing errors. These participants made the errors/mistakes during the keying in of their personal data while completing task 2 (this task required participant’s personal banking data in order to reload their eWallet). The errors are categorized as non-critical and do not prevent successful completion of the task. The errors are just about user typing errors while they are trying to key in the data. Participants can make correction easily and proceed to complete the desired task. Table 4 shows the task errors per participants. Table 4. Number of mistakes made while participants keyed in data P1 P2 P3 P4 P5 P6 P7 P8 P9 P10 P11 P12 P13 P14 P15 Task 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Task 1 Task 2 3 0 0 0 0 0 1 0 0 0 0 2 0 2 0 Task 2 Task 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 Task 3 Task 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Task 4 iJIM ‒ Vol. 15, No. 06, 2021 9 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App Task Ratings: After the completion of each task, participants rated the interface de- sign, ease of use and their experience based on the following nine criteria: i) Satisfaction with application interface - interface design; ii) Satisfaction with the use of text - inter- face design; iii) Satisfaction with the use of colour - interface design; iv) Satisfaction with the graphics and sound - interface design; v) Satisfaction with the menu provided - ease of use; vi) Easy to find help - ease of use; vii) Satisfaction with the output - experience assessment; viii) Satisfaction with the application performance - experience assessment; and ix) Enjoy the application? - experience assessment. For this test, nine items were used based on 5-points Likert scale that ranged from 1. (“Strongly disa- gree’’) to scale 5. (“Strongly agree’’). Agree ratings combines the agree ratings and strongly agree ratings. a) Interface Design Application interface: All participants agreed they are satisfied with the application interface for task 1 (“Check Account Balance” with mean agreement rating = 5.00). Almost all the participants (93.3%) were satisfied with the interface during task 2 (“Re- load Money” with mean agreement rating = 4.47). In task 3 (“Perform Money Trans- fer”), 93.4% of participants liked the interface (mean agreement rating = 4.27) mean- while everyone were satisfied with the application interface in task 4 (“Perform pay- ment at selected attached merchants”, mean agreement rating = 4.33). Use of text: All participants agreed they are satisfied with the use of text in task 1 (“Check Account Balance”, mean agreement rating = 5.00) and 93.3% of participants were satisfied with the use of text during task 2 (“Reload Money”, mean agreement rating = 4.33). In task 3 (“Perform Money Transfer”), all participants (100%) liked the use of text (mean agreement rating = 4.60) meanwhile everyone (100%) were satisfied with the use of text in task 4 (“Perform payment at selected attached merchants”, mean agreement rating = 4.27). Use of colour: All participants (100%) agreed they were satisfied with the use of color in all task. However, the mean task rating varies in each task. The mean task rating for task 1 is 5.00, that of task 2 is 4.60, the one of task 3 is 4.47 while that of task 4 is 4.53. Graphics and sounds: All participants (100%) in task 1, task 2 and task 3 agreed they are satisfied with the graphics and sounds of the app with mean task ratings of 5.00, 4.47 and 4.73 respectively. However, in task 2, 6.7% of participants were dissatisfied with the graphics and sounds of the application with mean task rating of agreement of 4.27. Table 5 provides a summary of the users’ satisfaction perception of the app’s interface design. Overall, the mean interface design satisfaction perceptions are more than 4.00. Thus, it is considered that the users were satisfied with the interface design of the application. Table 5 shows the mean interface design satisfaction ratings per task for the application. 10 http://www.i-jim.org Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App Table 5. Interface Design Mean Satisfaction Perception Ratings per Task Task Application interface Use of text Use of colour Graphics and sounds Over- all 1. Check Account Balance 5.00 (100%) 5.00 (100%) 5.00 (100%) 5.00 (100%) 5.00 2. Reload Money 4.47 (93.3%) 4.33 (93.3%) 4.60 (100%) 4.27 (93.3%) 4.42 3. Perform money transfers 4.27 (93.4%) 4.60 (100%) 4.47 (100%) 4.47 (100%) 4.45 4. Perform payment at selected attached merchants 4.33 (100%) 4.27 (100%) 4.53 (100%) 4.73 (100%) 4.47 Percent Agree (%) = Agree & Strongly Agree Responses combined b) Ease of use Menu Navigation: Task 1 has the highest mean task rating with 4.80. Almost all participants (93.3%) were satisfied with ease of menu navigation of the application. Same percentage of participants perceived the menu navigation in task 2 as easy to use (mean agreement rating = 4.40). Meanwhile, 87.6% of participants were satisfied with the menu navigation ease in both task 3 and task 4 (“money transfer and payment”, mean agreement rating = 4.13). Use of text: 40.0% of the participants were satisfied with the ease of finding help while performing money transfer (mean agreement rating = 2.80) and only 26.7% found it is easy to find help when reloading money (mean agreement rating = 4.33); the mean agreement rating declined to 1.87 where only 13.3% of participants found it easy to find help while checking account balance. However, none of the participants could find help easily during task 4 while performing payment at selected attached merchants (mean agreement rating = 1.60). Overall, in the ease-of-use satisfaction ratings; mean agree- ment ratings for all tasks are less than 4.00. Thus, it is considered that the users do not think this application is easy to use. Table 6 shows the mean ease of use perception ratings per task for the application. Table 6. Ease of Use Mean Perception Rating per Task Task Menu Navigation satisfactory Easy to find help Overall 1. Check Account Balance 4.80 (93.3%) 1.87 (13.3%) 3.34 2. Reload Money 4.40 (93.3%) 2.80 (26.7%) 3.60 3. Perform money transfers 4.13 (86.7%) 2.80 (40.0%) 3.47 4. Perform payment at selected attached merchants 4.13 (86.7%) 1.60 (0%) 2.87 Percent Agree (%) = Agree & Strongly Agree Responses combined c) Experience Assessment Output: All participants agreed they were satisfied with the output for task 1 (“Check Account Balance”, mean agreement rating = 5.00) and that of task 2 (“Reload Money”, mean agreement rating = 4.67). In task 3 (“Perform Money Transfer”), 80.0% of participants were satisfied with the output (mean agreement rating = 3.80) meanwhile all participants (100%) were satisfied with the output in task 4 (“Perform payment at selected attached merchants”, mean agreement rating = 4.40). Performance: 100% of the participants were satisfied with the performance of appli- cation while performing money transfer (mean agreement rating = 5.00) and 93.4% of iJIM ‒ Vol. 15, No. 06, 2021 11 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App them found the performance of application satisfactory when reloading money (mean agreement rating = 4.33); the mean agreement rating of performance satisfaction while checking account balance is 4.67 with 100% participant agreement. The participants had the same level of satisfaction perception for task 4 while performing payment at selected attached merchants (mean agreement rating = 4.40). Enjoyment: All participants enjoyed carrying out task 1 (“Check Account Balance”) and task 2 (“Reload Money”) with mean agreement rating = 5.00. In task 3 (“Perform Money Transfer”), 100% of participants so much enjoyed using the application (mean agreement rating = 4.53), similarly, all participants enjoyed carrying out task 4 (“Per- form payment at selected attached merchants”, mean agreement rating = 4.47). Table 7 shows the mean experience assessment ratings per task. Overall, the mean experience assessment ratings for all tasks are more than 4.00. Thus, it is considered that the users enjoyed using this application. Table 7. Mean Experience Assessment Ratings per Task Task Output Performance Enjoyment Overall 1. Check Account Balance 5.00 (100%) 5.00 (100%) 5.00 (100%) 5.00 2. Reload Money 4.67 (100%) 4.33 (93.4%) 5.00 (100%) 4.67 3. Perform money transfers 3.80 (80.0%) 4.67 (100%) 4.53 (100%) 4.33 4. Perform payment at selected attached merchants 4.40 (100%) 4.40 (100%) 4.47 (100%) 4.42 Percent Agree (%) = Agree & Strongly Agree Responses combined Table 8 shows the overall mean ratings of the application per task. In all tasks, the average satisfaction was above 4.00 except for task 4 with a rating of 3.92. Nonetheless, there was a good performance rating in all tasks indicating that overall, the perceived instrumental quality of the application is good. Table 9 summarizes all the metrics cap- tured in the study per task. Table 8. Overall Mean Task Ratings per Task Task Interface Design Ease of use Experience Assessment Overall 1. Check Account Balance 5.00 3.34 5.00 4.45 2. Reload Money 4.42 3.60 4.67 4.23 3. Perform money transfers 4.45 3.47 4.33 4.08 4. Perform payment at selected attached merchants 4.47 2.87 4.42 3.92 Table 9. Summary of Completion, Errors, Time on Task, Mean Satisfaction Task Task Completion Errors Time on Task Satisfaction* 1 15 0 16.40 4.45 2 15 8 150.13 4.23 3 15 1 192.00 4.08 4 15 0 35.46 3.92 *Satisfaction = Mean combined rating across four task measures 12 http://www.i-jim.org Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App Table 9 indicates the mean combined ratings across the four task measures. The rat- ings across the four task measures are satisfactory and show that the app has good in- strumental quality overall. 4 Issues and Recommendations Fifteen (15) participants participated in testing the Touch ‘n GO eWallet mobile ap- plication. Though overall the instrumental quality of the app is, however, a close scru- tiny reveals some issues for require redesign. The major problems that were discovered based on the data collected during the testing and evaluation are: i. Icons: The significance of the icons was unclear for some test participants. These icons were tightly placed which made it more difficult to distinguish them. Some of the icons for some participants were not suitable for the task given. Participant 8 felt that the icon of money transfers were not clear enough to represent the task. ii. Text: Some views were missing explanatory text which led to uncertainty amongst participants. Most of the participants felt that task 1: ‘Check account balance’ was hard to find. Using text for checking equilibrium in these appli- cations is quite tiny. Likewise, the fonts are tiny. The font's color is white; the blue background is quite a reflection. iii. Help instructions: Help instructions cannot be found throughout the entire task. Most respondents were unable to discover the help features they needed in any task. The introductory instruction only took place at the start of the applications setup. iv. Action order: Before tasks for money transfer executed, participants had to submit identification for security purpose, in order to do that, the task of cap- turing ID has to be fulfilled and executed, there should be caution in this task as it may take a long time when the capturing of images did not meet the spec- ifications of the applications such as blurry picture, and picture reflection. If it takes a long time to recover the picture, the user may get bored and frustrated. The retake of picture was done approximately more than 3 times. Participants must use some techniques like turning off the lights and adjusting the picture capture angle. Most of the respondents experienced this (Participant 1, 2, 3, and 6). Conversely, some features were so greatly favored by participants as captured in the evaluation: i. Navigation: The participants felt that navigation in the app was easy and most of the participants were very happy when they completed the tasks success- fully. ii. Other Useful Features: Other features that interested many of the participants were the promotions given by the apps like Cashback for Toll, Get RM5 off any movie Ticket, PayDirect for 20% Toll Rebates, and much more. Although iJIM ‒ Vol. 15, No. 06, 2021 13 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App these tasks are not included in the list of tasks for the study, provision of more rebates may serve as incentives to lure more customers to join and use the ap- plication. iii. Recommendations: Performing continuous usability testing during production on an app design can save a great deal of time, effort, and money. It may also guarantee a better design that could have a significant effect on boosting app usage. Most participants appreciated the characteristics provided for the usa- bility testing. Most respondents were between the ages 30 - 40, so age variables may likely have altered their perception of the applications; also, most partici- pants indicated that the font sizes for important characteristics should be ex- panded due to the tiny size of the smartphone and the tiny font size which they can hardly see in a glance. Often, they had to scroll up and down and forth to find the required feature. Next, a clear “help” feature should be provided in the apps. The only “help” feature was presented during the first installation and act as an introduction only. Participants were not able to discover the “help” after that. During the course of using the app, a help feature should be provided. 5 Conclusion Since there are many eWallet applications in the store today, owners need to discover a function that draws consumers to use their app at the highest possible rate. Such apps should be interesting and should function correctly. Having too many features in an implementation can sometimes result in a catastrophe. The idea of Touch ‘n Go eWallet is well conceived and timely as it is commonly implemented throughout Malaysia, so customers are requesting a shift from traditional Touch ‘n Go cards to eWallet Touch ‘n Go apps. The eWallet should drive and attract users in using the app and the design should be clear, clean and with minimalist display of features. This is the most im- portant issue that should be dealt with in order for the app to work with people who have no previous e-wallet experience. When testing, it was realized that user were quite slow in comprehending the app's design because it was their first experience, but when they got used to playing with it, it got faster and quicker. On average, the findings ac- quired from the evaluation exceed 85% of each allocated task from the experiments performed. Thus, the eWallet basic function (Task 1 until Task 4) is accepted by par- ticipants involved in the testing session. This study used 4 tasks to assess the instru- mental quality of Touch ‘n Go eWallet mobile app using 15 participants. Overall, the outcome of the study revealed that the app is generally instrumental to assisting users accomplishing their electronic wallet goals. There is however some observed issues in the app that require fixing to enhance the instrumental quality of app. 6 References [1] Dumas, J. S., & Redish, J. (1999). A practical guide to usability testing. Intellect books. [2] https://medium.com/@satrya.rudana/usability-testing-1-t-wallet-ca71a176431f 14 http://www.i-jim.org https://medium.com/@satrya.rudana/usability-testing-1-t-wallet-ca71a176431f Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App [3] Hussain, A., Mkpojiogu, E.O.C., Abubukar, H., & Hassan, H.M. (2019). A mobile usability test assessment of an online shopping application. Journal of Computational and Theoretical Nanoscience.16(5-6), 2511-2516. https://doi.org/10.1166/jctn.2019.7923 [4] Dray, Susan; Siegel, David (March 2004). "Remote possibilities? :international usability testing at a distance". Interactions. 11 (2): 10–17. https://doi.org/10.1145/971258.971264. [5] Hussain, A., Mkpojiogu, E.O.C., Faromiki, J.O., Abdusalam, E.M.A., & Shamala, P. (2019). Determining the comfort of use of a mobile navigation app. Journal of Advanced Research in Dynamical and Control Systems (JARDCS), 11(08-SI), 1144-1146. [6] Gafni, R. (2009). Usability Issues in Mobile-Wireless Information Systems. In Proceedings of the 2009 InSITE Conference (Vol. 6, pp. 1–15). https://doi.org/10.28945/3383 [7] Hussain, A., Mkpojiogu, E.O.C., & Suleiman, K. (2019). A heuristic evaluation of Achik.biz mobile shopping app. International Journal of Recent Technology and Engineering (IJRTE), 8(2S2), 123-126. https://doi.org/10.35940/ijrte.b1023.0782s219 [8] Gough, D., & Phillips, H. (2003). "Remote Online Usability Testing: Why, How, and When to Use It". Retrived on Aug 18, 2019. [9] Breuch, L.-A., Mark Z., & Spinuzzi, C. (April, 2001). "Usability Instruction in Technical Communication Programs". Journal of Business and Technical Comm. 15 (2), 223– 240. Retrieved 13Aug 2019. https://doi.org/10.1177/105065190101500204 [10] Hussain, A., Mkpojiogu, E.O.C., Ishak, N., Mokhtar, N., & Ani, Z.C. (2019). An interview report on users’ perception about the usability performance of a mobile e-government appli- cation. International Journal of Interactive Mobile Technologies (IJIM) 13 (10), 169-178. https://doi.org/10.3991/ijim.v13i10.11286 [11] Duh, H. B.-L., Tan, G. C. B., & Chen, V. H. (2006). Usability evaluation for mobile device. In the 8th conference on Human-computer interaction with mobile devices and services (pp. 181–186). https://doi.org/10.1145/1152215.1152254 [12] Hussain, A., Mkpojiogu, E.O.C., & Suleiman, K. (2019). A mobile usability testing of Achik.biz application. Journal of Advanced Research in Dynamical and Control Systems (JARDCS), 11(08-SI), 1024-1028. [13] Kaikkonen, A., Kekäläinen, A., Cankar, M., Kallio, T., & Kankainen, A. (2009). Evaluation of user interface design and input methods for applications on mobile touch screen devices. Lecture Notes in Computer Science, 243–246. https://doi.org/10.4018/978-1-59904-871- 0.ch053 [14] Hussain, A., Mkpojiogu, E.O.C., Ishak, N., & Mokhtar, N. (2019). A study on the perceived mobile experience of MyEg users. International Journal of Interactive Mobile Technologies (IJIM), 13 (1), 4-23. https://doi.org/10.3991/ijim.v13i11.11306 [15] Arslan, B., & Fröjdh, B. (2019). E-wallet - designed for usability (Dissertation). Retrieved from [16] http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-252790 [17] Jerz, D.G. (2000). "Usability Testing: What Is It?". Jerz's Literacy Weblog. Retrieved Aug 16, 2019. [18] Hussain, A., Mkpojiogu, E.O.C., Jamaludin, S., Hilaluddin, K., & Nathan, S. (2019). Users’ perception of their satisfaction and experience on a mobile cinema application, International Journal of Recent Technology and Engineering (IJRTE), 8(2S2), 131-134. https://doi. org/10.35940/ijrte.b1025.0782s219 [19] Andreasen, M.S., Nielsen, H.V., Schrøder, S.O., & Stage, J. (2007). What happened to re- mote usability testing? Proceedings of the SIGCHI Conference on Human Factors in Com- puting Systems - CHI '07. p. 1405. ISBN 9781595935939. https://doi.org/10. 1145/1240624.1240838 iJIM ‒ Vol. 15, No. 06, 2021 15 https://doi.org/10.1166/jctn.2019.7923 https://doi.org/10.1145/971258.971264 https://doi.org/10.28945/3383 https://doi.org/10.35940/ijrte.b1023.0782s219 https://web.archive.org/web/20051215231619/http:/www.boxesandarrows.com/view/remote_online_usability_testing_why_how_and_when_to_use_it https://web.archive.org/web/20051215231619/http:/www.boxesandarrows.com/view/remote_online_usability_testing_why_how_and_when_to_use_it http://jbt.sagepub.com/content/15/2/223.full.pdf+html http://jbt.sagepub.com/content/15/2/223.full.pdf+html https://doi.org/10.1177/105065190101500204 https://doi.org/10.3991/ijim.v13i10.11286 https://doi.org/10.1145/1152215.1152254 https://doi.org/10.4018/978-1-59904-871-0.ch053 https://doi.org/10.4018/978-1-59904-871-0.ch053 https://doi.org/10.3991/ijim.v13i11.11306 http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-252790 http://jerz.setonhill.edu/writing/technical-writing/usability-testing/ https://doi.org/10.35940/ijrte.b1025.0782s219 https://doi.org/10.35940/ijrte.b1025.0782s219 https://en.wikipedia.org/wiki/International_Standard_Book_Number https://en.wikipedia.org/wiki/Special:BookSources/9781595935939 https://doi.org/10.1145/1240624.1240838 https://doi.org/10.1145/1240624.1240838 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App [20] Virzi, R. A. (1992). "Refining the Test Phase of Usability Evaluation: How Many Subjects is Enough?". Human Factors. 34 (4): 457–468. https://doi.org/10.1177/0018720892034 00407 [21] Hussain, A., Mkpojiogu, E.O.C., Karmal, F.M., & Lateef, H.M. (2019). Ascertaining the UX of the word mania mobile app for children using fun toolkit v3. International Journal of Recent Technology and Engineering (IJRTE), 8(2S2), 202-205. https://doi.org/10.35940/ ijrte.b1037.0782s219 [22] Zhang, D., Adipat, B., & Zhang, D. (2005). Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications Challenges, Methodologies, and Issues in the Us- ability Testing of Mobile Applications. International Journal of Human-Computer Interac- tion, 18(3), 293–308. https://doi.org/10.1207/s15327590ijhc1803_3 [23] Hussain, A., Mkpojiogu, E.O.C., Adetunmbi, S., Hassooni, M.N., Basri, S. (2019). An eval- uation of the effectiveness in use and efficiency in use of a mobile news aggregator magazine application. International Journal of Innovative Technology and Exploring Engineering, 8(8S), 527-531. [24] Rubin, J., & Chisnell, D. (2008). How to plan, design, and conduct effective tests. Handbook of usability testing. [25] Balagtas-Fernandez, F., & Hussmann, H. (2009). A methodology and framework to simplify usability analysis of mobile applications. ASE2009 - 24th IEEE/ACM International Confer- ence on Automated Software Engineering, 520–524. https://doi.org/10.1109/ase. 2009.12 [26] Nadzir, M., Hussain, A., Mkpojiogu, E.O.C., Faromiki, J.O., & Abdusalam, E.M.A. (2019). The effectiveness and efficiency of a GPS route and voice navigation app, International Journal of Innovative Technology and Exploring Engineering, 8 (8S), 425-427. [27] Nielsen, J. (1994). Usability Engineering, Academic Press Inc, p 165 [28] Nadzir, M., Hussain, A., Mkpojiogu, E.O.C., Adetunmbi, S., & Hassooni, M.N. (2019). As- sessing the comfort in use of a flipboard mobile news magazine app. International Journal of Innovative Technology and Exploring Engineering, 8(8S), 428-430. [29] Paternò, F., Russino, A., & Santoro, C. (2007). Remote evaluation of mobile applications. In International Workshop on Task Models and Diagrams for User Interface Design (pp. 155–169). https://doi.org/10.1007/978-3-540-77222-4_13 [30] Hussain, A., Mkpojiogu, E.O.C., Tam, K.L., Hilaluddin, K., & Jamaludin, S. (2019). A measure of the effectiveness and efficiency of an m-cinema app. International Journal of Recent Technology and Engineering (IJRTE), 8(2S2), 127-130. [31] Ryu, Y. S. (2005). Development of Usability Questionnaires, (July), 1–206. [32] Technologie, É. De, & Université, S. (2012). The State of the Art of Mobile Application Usability Evaluation Fatih Nayebi, Jean-Marc Desharnais , Alain Abran. In In 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE) (pp. 1–4). https://doi.org/10.1109/ccece.2012.6334930 [33] Hussain, A., Mkpojiogu, E.O.C., Nabeel, N., & Alathwari, A. (2019). Users perception of the mobile usability of a global bicycle sharing platform. International Journal of Interac- tive Mobile Technologies (IJIM), 13 (1), 125-136. https://doi.org/10.3991/ijim.v13i11. 11298 [34] Jauhar, J., Ghani, A.B.A., Joarder, M.H.R., Subhan, M., Islam, R. (2015). Brain Drain to Singapore: A Conceptual Framework of Malaysians' Diaspora. Social Sciences (Pakistan), 10 (6), pp. 702-711. 16 http://www.i-jim.org https://doi.org/10.1177/001872089203400407 https://doi.org/10.1177/001872089203400407 https://doi.org/10.35940/ijrte.b1037.0782s219 https://doi.org/10.35940/ijrte.b1037.0782s219 https://doi.org/10.1207/s15327590ijhc1803_3 https://doi.org/10.1109/ase.2009.12 https://doi.org/10.1109/ase.2009.12 https://doi.org/10.1007/978-3-540-77222-4_13 https://doi.org/10.1109/ccece.2012.6334930 https://doi.org/10.3991/ijim.v13i11.11298 https://doi.org/10.3991/ijim.v13i11.11298 Paper—An Instrumental Assessment of Touch ‘n Go eWallet Mobile App 7 Authors Azham Hussain is the Associate Professor of Software Engineering at School of Computing, Universiti Utara Malaysia, 06010 UUM, Sintok, Kedah, Malaysia. He is the founder of Human-Centered Computing Research Group, which is affiliated with the Software Technology Research Platform Center at School of Computing, Universiti Utara Malaysia. Azham Hussain is a member of the US-based Institute of Electrical and Electronic Engineers (IEEE), and actively involved in both IEEE Communications and IEEE Computer societies. hussainazh1217@gmail.com Emmanuel O.C. Mkpojiogu is a Lecturer at Department of Computer and Infor- mation Technology, Veritas University, Abuja, Nigeria. Currently, he is a PhD student at School of Computing, Universiti Utara Malaysia, 06010 UUM, Sintok. The research area is User Experience, Human Computer Interaction and Software Engineering. He has published many articles in reputable Scopus indexed journals. emelnuel@hot- mail.com Fazillah is a lecturer at School of Quantitative Sciences, Universiti Utara Malaysia, 06010 (UUM). She receives her Bachelor of Information Technology (BIT) in Com- puter Network from the same university she is currently affliated, and M.Sc. of Com- puter Science in Multimedia from Universiti Putra Malaysia (UPM). She has presented and published papers and articles in several prestigious conferences and journals, and taught courses related to her field. Apart from these, she actively conducts research in Human – Computer Interaction (HCI) and put much interests on interaction design and technology design focusing on collaborative aspect of the interaction which is deemed imperative in decision-making process. At present, she is a PhD candidate at Universiti Teknologi MARA (UiTM) and completing her PhD thesis dealing with the social-tech- nical aspects of social messaging systems that will inform future design of connecting technology for domestic use. fazillah@uum.edu.my Rohazna Wahab (School of Computing, Universiti Utara Malaysia, 06010 UUM, Sintok, Malaysia) is an IT officer at Universiti Malaysia Perlis. Currently, she is doing Master in Information Technology at Universiti Utara Malaysia. rohazna_wahab1 @ahsgs.uum.edu.my Noor Halawati Che Meh (School of Computing, Universiti Utara Malaysia, 06010 UUM, Sintok, Malaysia) is a lecturer at Politeknik Tuanku Syed Sirajuddin. She inter- ested to do research on software engineering and mobile interface. noor_halawati_ che@ahsgs.uum.edu.my Article submitted 2020-12-18. Resubmitted 2021-01-24. Final acceptance 2021-01-26. Final version pub- lished as submitted by the authors. iJIM ‒ Vol. 15, No. 06, 2021 17 mailto:hussainazh1217@gmail.com mailto:emelnuel@hotmail.com mailto:emelnuel@hotmail.com mailto:fazillah@uum.edu.my mailto:rohazna_wahab1@ahsgs.uum.edu.my mailto:rohazna_wahab1@ahsgs.uum.edu.my mailto:noor_halawati_che@ahsgs.uum.edu.my mailto:noor_halawati_che@ahsgs.uum.edu.my