International Journal of Interactive Mobile Technologies(iJIM) – eISSN: 1865-7923 – Vol 16 No 09 (2022) Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved Safety and Ergonomics in Real Driving Scenario https://doi.org/10.3991/ijim.v16i09.30261 Jayasankari Ganasan(), Ahmad Sobri Hashim Universiti Teknologi PETRONAS (UTP), Seri Iskandar, Perak, Malaysia jayasankari_20000248@utp.edu.my Abstract—Mobile phones that accept input by a user’s finger motion are becoming increasingly common. However, they still have challenging problems to research, such as the ergonomic and safety aspects. As an outcome, more research required its capabilities and a critical examination of the existing ges- tural interfaces and how they assist drivers’ activities while driving. This research paper attempts to present a comprehensive understanding of possible gestures on smartphones by conducting user testing with sample data of 30 drivers from three different age groups. The user testing was performed in an actual driving envi- ronment. Observation and interviews were carried out to study drivers’ behavior while driving. The data gathered were then interpreted into action and motivation levels. The action level was defined by how drivers interact with smartphones while driving, and in motivation level, a study on why drivers interact in that manner was conducted. The results were then drawn to a table with the task car- ried out during user testing. In conclusion, this research aims to consider all these drivers’ issues while driving. This is to determine how a more advanced gestural interaction of smartphone interfaces may be created to meet drivers’ safety and ergonomic concerns. Keywords—action level, car driver, finger gesture, gestural interactions, motivation level, smartphone user interface 1 Introduction Currently, gestural interaction is a dominant research area in the human-computer interaction field. It has also been used in various areas such as mobile learning [1]. Previous research on gestural interaction has concentrated on gesture definition and recognition. As stated in [2], a gestural interface can detect and recognize motions rapidly and accurately. As a result, a gestural interface is considered a possible solution to let drivers engage with their smartphones while driving. Because it is challenging to keep drivers away from constantly using their smartphones, this solution could help meet the existing demand. However, some issues need to be addressed in gestures to provide consistency for car drivers. 76 http://www.i-jim.org https://doi.org/10.3991/ijim.v16i09.30261 mailto:jayasankari_20000248@utp.edu.my Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… The use of smartphones while driving has risen substantially in recent years, notably with the proliferation of social media mobile apps [3][4]. Mobile phones are the leading cause of driver distraction, according to a study conducted by Ref. [5]. They have an impact on driving performance and enhance the likelihood of an accident. Drivers continue to interact with their smartphone user interface despite the possible dangers of doing so while driving. It is usually considered for a driver to check and respond to incoming smartphone notifications while driving. Driving is one of the most common activities during which people use their smart- phones as a secondary task, according to Ref. [6]. The underlying reason was that driv- ers were comfortable and used to engaging with smartphones while driving until the unpleasant incident takes place. Due to this day-to-day behavior of drivers, the fre- quency of car accidents continues to rise as drivers continue to disobey the law prohib- iting the use of smartphones in certain situations, such as while driving [7]. As a result, developing a gestural interaction model is critical to allow automobile drivers to engage with smartphone user interfaces while driving. In the long run, this concept could reduce the number of car accidents caused by smartphone use. User tests were conducted with car drivers to gather all the drivers’ gestures that support drivers’ driving conditions, driving in confined space with limited distraction. This experiment demonstrates how drivers interact with the smartphone and why would they perform those interactions in that manner. In summary, this test is concerned with the available gestures on smartphones, and drivers respond to each of the tasks carried out. The section in this paper includes, Section 1 provides an introduction of the work, Section 2 discusses the most relevant literature review and summarizes all the related work, Section 3 explains the method- ology and Section 4 presents the results followed by the discussion. Finally, Section 5 concludes the research with some future ideas. 2 Related work Non-verbal communication between a human and a system or equipment meant to transmit a specific message refers to as a “gesture” [8]. Gesture recognition is a term used to describe technologies that allow electronic devices to control by a person’s hands, fingers, head, body, or any other physical movement that can be translated [9]. The basic idea behind gesture recognition is that a computer can understand and execute commands based on gestures. One or more optical sensors uses to capture an individual’s images, and advanced software interprets the images to identify the human action. Gesture recognition serves real-time data to a computer as an alternative user interface [10]. Rather than typing with keys or tapping on a touch screen, a motion sensor detects and interprets gestures as the primary data input source. Human move- ments identify as part of a more extensive system of pattern recognition. In this para- digm, there are two phases: the representation process and the decision process [11]. The representation approach converts raw numerical data into a classification-based decision-making form. The path depicts in Figure 1. iJIM ‒ Vol. 16, No. 09, 2022 77 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Fig. 1. The general structure of gesture recognition system Gesture recognition systems inherit this structure and add two other processes: acquisition and interpretation. The acquisition process converts physical gestures into numerical data, while the interpretation process lends meaning to the sequence of sym- bols generated by the decision process [12][13]. A similar study was conducted by Ref. [14] on the impact of different smartphone navigation settings and modes on real-world driving behavior. Twenty professional drivers took part in the user testing conducted in free-flowing traffic and good weather conditions. The evaluation was done with the eye movement and vehicle control data gathered from the experiment. The experiment results show that a smartphone naviga- tion device placed on the right side of the car dashboard (Position 1) has less impact on driving behavior than when placed above the air conditioning vent (Position 2). A smaller angle of view can increase the fixation frequency and the length of time the driver spends looking out the windshield and reduce the range and time spent glancing at the navigation device. Figure 2 illustrates the layout of the experimental setup. Fig. 2. Position of smartphone [14] Apart from that, as stated in [15], works similar to the current study. The goal of the research was to analyze if social networking on a smartphone impacted driving performance. In the study, twenty-eight young male and female volunteers drove a driving simulator through the same test scenario twice, once while using a smartphone to communicate with a social networking site and once without. The experiment’s outcome suggests that participants’ driving was impaired when using a smartphone to send and receive messages on a social networking site. These drivers encounter three significant distractions such as having to concentrate on the 78 http://www.i-jim.org Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… smartphone task (cognitive), holding the phone (manual) and the significant increase in time spent looking at the phone (visual) to interact with it. Even though the participants reduced their speed, they were still unable to control the vehicle effectively. Figure 3 illustrates the simulator driving setup. Moreover, a similar study was conducted by Ref. [16], to determine the number of drivers in Shah Alam, Selangor who were involved in distracting tasks while driving. There were two methods carried out to collect the data. First, questionnaires were dis- tributed among the drivers to study the distraction influenced by the age and gender factor. Next, a real-life observation was conducted in various locations by capturing the driving activities from the upper view. The data collected was then used to analyze the differences between age, gender, and peak and non-peak hours. Fig. 3. TRL driving simulator, DigiCar [15] The observation took place in three selected locations: the major road in the Shah Alam area that consists of three lanes in Seksyen 7, Seksyen 19 and Seksyen 21. The result shows that males are more distracted with smoking habits than females, peak hours show that drivers tend to be distracted more than a non-peak hour, and younger drivers tend to be distracted by smartphones compared to elderly drivers. The camera and the observer were set to place at the upper view (roads crossing using the bridge), as shown in Figure 4. One of the observation locations (Seksyen 7) is shown in Figure 5, which was a random selection with the testing of availability of upper pedestrian pass and the availability of vehicles passed by. The location for the observer was marked with a red line. Fig. 4. Camera positioning for observation study [16] iJIM ‒ Vol. 16, No. 09, 2022 79 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Fig. 5. The location in Seksyen 7 for observation [16] Hence, the data adapted summarizes various methods to carry out a similar study apart from what will be carried out in this research. But there are still some missing factors that are yet to be addressed. So, in this research, we have encountered those and made an extensive analysis to develop the method to conduct this user testing with car drivers. Description of how this study was carried out is discussed in the methodology section of this paper; results of the user testing are discussed in the section after that. 3 Methodology Based on the background of the study, it proves that driving is one of the top activi- ties during which people use smartphones as a secondary task [16]. Using a smartphone while driving is one of the six primary causes of road accidents in Malaysia, as stated in [17]. These accidents are approximately 1.5 M, about 26% of the total road crash cases [18]. Thus, this study has evinced that the implementation of available technolog- ical solutions did not translate to the required decrease in car crash cases. Drivers are prone to ergonomic and safety issues during an interaction with the smartphone while driving [19]. The study has also concluded a lack of understanding of drivers’ mental models as they drive and parallelly interact with smartphones. Hence, through extensive investigation and exploration of the existing gestural inter- face, the outcome has identified that an effective and more advanced gestural interface is a potential solution to the recent spike in car crash cases [20][21]. Even though the current law forbids using a smartphone while driving, it is almost impossible to omit entirely. Therefore, this will be a solution to rather minimize taking drivers’ atten- tion when mobile devices must be used. In future, the presented test cases will assist smartphone developers in focusing more on the core aspects of drivers’ safety and ergonomics. This research has adopted a three-phase approach which includes explorations, for- mulation and finally, validation. This paper will address phase one which is, identifying the interaction elements, restricting the focus on the user test on drivers’ interactions when interacting with the smartphone user interface. 3.1 Experiment setup This section details the research process and activities carried out at the phase of interaction element identification, specifically the user test on drivers’ interactions 80 http://www.i-jim.org Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… when interacting with the smartphone user interface. The flow chart as presented in this section represents each step carried out. This flow chart is shown in Figure 6. Fig. 6. Summarization of research methodology As depicted in Figure 6, these steps are further elaborated as follows. The user test- ing involved 30 drivers from three different age groups demographically: 18–35 years old, 36–50 years old and more than 50 years old, where ten drivers in the respective category. The main objective of conducting user testing with various age groups is to identify, compare, and then analyze the way of interaction with the smartphone while driving and what makes them perform in that manner [22]. Drivers’ credentials are kept anonymous under the privacy and confidential term. They are given the flexibility to have familiarization drive up to 10 minutes to get used to the experimental car. Tables were tabulated according to age category which consist of drivers’ ID, age, gender, and the need for familiarization drive before the user experiment. The user testing was performed under certain constant variables on the design and procedure to ensure the relevancy of the outcome across the age categories [23]. The variables are scope down to meet the research objective. First and foremost, user testing is performed in an actual driving environment. Hence, the targeted geographic location was an urban area, specifically Ipoh, Perak. The reason behind choosing a particular area apart from the scope of research is the literature study that details a high number of car crash cases in those areas [24]. The road track was fixed throughout the experiment. The experimental car was set to be the same for all the drivers; thus, Honda HR-V, an auto-car, would be the one. Auto- car was chosen to ease the driver’s experience while conducting user testing and more focused on the required research outcome. The car’s speed sets to be 40–60 km/h for all the drivers throughout the user testing; this is to ensure the safety of drivers [25]. The user testing was carried out during the daytime between 10 am to 7 pm. Since cameras iJIM ‒ Vol. 16, No. 09, 2022 81 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… were mounted in the car during the user testing, daytime has been an excellent option to ensure a precise clarity of video recording. Furthermore, the user testing on smartphone gestural interface centred to Android operating system. This is to scope the data gathering process and focus on a respective problem-solving area. The drivers are free to use their smartphones during the experi- ment to meet the requirement stated. This will be more casual and stress-free for drivers and grasp the exact interaction with a smartphone [26]. The drivers were instructed to perform frequent tasks while driving, such as making and answering calls, sending, and reading messages, and browsing GPS navigation. Then, several notable reminders were given to the drivers to ensure smooth and well-planned user testing. Once the drivers get into the car, the experimenter makes sure they buckle up their seat belts. Drivers’ safety is the utmost priority. They were reminded hours before the experiment to ensure the phone’s battery power is sufficient since it takes up to 30 minutes to complete the user testing for each driver. If the driver requires a phone holder or any additional equipment to place the phone while driving, he/she must inform the experimenter beforehand. Since the phone’s screen recording will be done, the drivers are advised not to expose their details or any confidential infor- mation. After a clear understanding and agreement of drivers towards the experiment procedures, they got to place their signature on the consent form provided. Next, the setting up of a usability test was done. There were two cameras mounted in the car to record the live video. The front camera (Figure 7) was mounted at the windscreen facing the driver. The purpose of this camera was to capture the eye move- ment of drivers while performing tasks. This is a measure of drivers’ distraction while driving as it records the eyes-off road duration. The second camera was mounted at the driver-side window (Figure 8). The purpose of this camera was to record the finger gesture of drivers on the smartphone interface while performing tasks. This setup has allowed drivers to have a free and casual driving environment, parallelly recording the natural interaction of drivers with the smartphone. The layout of the experimental setup described earlier has been delineated in Figures 7 and 8. Fig. 7. Camera mounted at windscreen (Front view) 82 http://www.i-jim.org Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Fig. 8. Camera mounted at the driver-side window (Side view) Together with user testing, the experimenter has recorded all the necessary and observed movements or gestures of drivers while driving. Each driver has got their observation form. The data recorded includes the smartphone’s placement while driv- ing. The drivers are free to place their smartphones at any preferred location in the car. As mentioned, frequently performed tasks have been carried out and recorded during the experiment. However, it is not limited as drivers are given the flexibility to use smartphones as they do regularly. This is to ensure drivers are not restricted to within certain boundaries [27]. Moreover, the type of distraction records for each task performed. Distraction is categorized into four major types. Visual distraction indicates drivers look away from the roadway to obtain specific information visually [28]. On the other hand, manual distraction shows drivers are doing any physical tasks parallelly, which is out of driving position [29]. An auditory distraction means the driver hears something from an exter- nal source and is unrelated to the driving task [30]. Finally, cognitive means drivers get distracted mentally while driving. Next, the level of findings was recorded. It divides into two categories which are action and motivation level. The action level explains how drivers interact with smart- phones using gestures later extracted from the video recording. In contrast, the motiva- tion level explains why drivers perform those gestures to fulfil the tasks on smartphones while driving, where the experimenter filled up via observation and interview after the experiment. Additionally, drivers’ positions were recorded manually upon the experiment began. This data later helps to study in the basis of ergonomics and safety, where the data gath- ered on smartphone placement also help. Apart from that, post user testing survey has been carried out with drivers. The driv- ers were questioned on the message received while driving. The objective of this test is to study the mental model analysis of drivers while driving. In this way, the survey can capture whether drivers remember the message read while driving; in short, do focus on the text read. 4 Results & discussion In this section, the result obtained from the user testing will be discussed. Once the user testing with 30 drivers from three different age groups was completed, the data iJIM ‒ Vol. 16, No. 09, 2022 83 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… gathered were further analyzed. As mentioned in the methodology section, user testing and observation have been carried out under two major levels, the action and motiva- tion level. As for the action level, the data was gathered from both cameras fixed in the car. The front camera (Figure 9) placed at the windscreen recorded drivers’ interaction with smartphones and eye movement, specifically the eyes off-road duration. The side cam- era (Figure 10), mounted at the driver-side window, captures the smartphone gestures while performing various tasks. Figures 9 and 10 show the placement of the front cam- era and side camera marked in the red border, respectively. Fig. 9. Mounting of front camera (At windscreen) Fig. 10. Mounting of side camera (At driver-side window) The front camera has recorded the duration of the number of glances at the phone screen while the driver performs a secondary task. Each age category holds separate data based on the task performed. The average duration will be identified at the end. This helps to know the distraction duration based on age group. After the analysis of data gathered in the number of glances at the phone screen while driving, it is concluded that drivers of age group 18–35 years old have got the highest distracted duration or glances as they perform secondary tasks while driving. The post-user testing interview with drivers supported the conclusion. The drivers of this age group preferred to engage in their smartphones parallelly driving on the road. 84 http://www.i-jim.org Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Thus, they got to focus on the phone and road simultaneously, leading to the most dis- tracted duration. On the other hand, drivers of more than 50 years old prefer to stop the car at the shoulder of the roadway while conducting any smartphone tasks. This supports by feed- back given as they feel much more comfortable engaging with their smartphone in this way. Moreover, they are not able to do both tasks simultaneously as the mental model analysis has proven the result. These drivers cannot concentrate or focus on smart- phones while driving; thus, they prefer to stop the car or never attend to the phone at all. The following data gathered upon user testing and observation were the driver distraction details. The experiment concluded that drivers aged 18–35 years old and 36–50 years old are much more distracted visually and manually, whereas drivers aged 50 and above are cognitively distracted. They are not able to concentrate when there is a call, notification, or an alert on their smartphone. This quickly diverts their concentra- tion off the road. The result also shows that no categories have got higher numbers in auditory distraction, which means the ringing tone, messaging alert and GPS navigating voice does not take drivers’ attention off-road. Furthermore, as smartphone placement was recorded manually during the user testing, the compiled result has shown that the younger age group 18–35 years old prefer to place their phone on their thigh (Figure 11) while driving. This supports the post-interview with drivers as they feel much more convenient, easy access and quick-reachable location. As mentioned, drivers of this age prefer to attend their smart- phones while driving, so placing the phone between thighs is convenient. Nevertheless, most drivers aged 36–50 years old prefer to place their smartphones at the driver-side door console (Figure 12). The drivers of age group 50 and above prefer to place their smartphone at the centre console (Figure 13), and most of them prefer to place their phone at the dashboard, especially when using GPS navigation. Hand-held is very much rare for this category of drivers. Fig. 11. Placement of smartphone on the thigh iJIM ‒ Vol. 16, No. 09, 2022 85 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Fig. 12. Placement of smartphone at a driver-side console Fig. 13. Placement of smartphone at the centre console Additionally, the drivers’ position while driving has been extracted throughout the experiment. The result has shown that most drivers of age group 18–35 years old have their pelvis back with seat, the head was near to the headrest, and the distance with the steering is leaning backwards. While on the contrary, most of the drivers of age group 36–50 years old choose to sit in the middle and distance with the steering are pretty at the mid place. Drivers of age group 50 and above structure their body position upright, and the head seems to be in front and quite close to the steering. This was supported by the explanation given by the drivers during post-user testing. Besides, the action level findings were derived from both the cameras. The action level addresses how drivers interact with smartphone tasks while driving. A table was drawn for each driver in each category for the smartphone tasks performed. The video recording has been used to detail how drivers used the smartphone for each of the tasks performed. Mainly all the smartphone user interface gestures were extracted from the video. A few sets of frequently performed gestures have been extracted from the pool of gestures for each of the tasks performed according to the age groups. The following tables point out the tasks performed, gestures and findings based on action and motiva- tion level for each category, respectively. 86 http://www.i-jim.org Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Table 1. Most frequently used gestures by age group 18–35 years old and findings of action and motivation level Age Group (18–35 years old) Tasks Gestures Action (How?) Motivation (Why?) Make call Tap, swipe up, slide right, scroll up and down Most drivers use tap gestures to select an app, enter a number or alphabet, select, etc. Swipe up is used at the lock screen to move the screen upwards and enter the password. Slide right is used to make a call by moving smoothly along the surface while maintaining constant touch on the contact. Scroll up and down is mainly used to search the caller at the frequently contacted list. Drivers also search for the initial at phonebook and scroll for the callers’ contact. Tap gesture is used at most of the steps before performing calling tasks to make various decisions. Swipe up brings the keyboard to the screen to enter the password; it helps drivers to enter passwords quickly. Slide right is used instead of tapping at contact. Scrolling up and down makes it easy to find the caller on the frequently contacted list, as it is easily accessible. Drivers do not have to enter all the alphabets to search for a contact; instead, they can scroll the phonebook up and down based on the initial alphabet. Answer call Swipe up, tap Swipe up is used to swipe the call icon upwards. Tap gesture is used to tap on speaker or hands-free icon if the driver decides not to hold the phone while driving. Tapping on the green icon allows incoming calls. Swiping up the green icon upwards allows incoming calls, and drivers can talk on the phone. Tap gesture mainly used after answering the call, to set as a loudspeaker. The driver also taps on the green icon to answer the call and talk on the phone. Send text message Tap, swipe up Mainly driver uses tap gesture to select an app, find the sender, type the message, and take other steps before sending the message. Swipe up is used at the lock screen to move the screen upwards and enter the password. Tap gesture is widely used at most steps before performing the messaging task to make various decisions. Swipe up brings the keyboard to the screen to enter the password. Read text message Tap, swipe up, swipe down The driver uses tap gesture to go message app, select the chat to read the message, sometimes to reply if necessary. Swipe up is used at the lock screen to move the screen upwards and enter the password. Swipe down is used to drag down the notification bar. Tap gesture is widely used to make a selection of conversation/ chat and read the text. Replying to the text also includes tap gestures. Swipe up brings the keyboard to the screen to enter the password. The driver can check the message at the notification bar without going to the messaging application. This way reduces the time and able to choose the important message and reply if needed. (Continued) iJIM ‒ Vol. 16, No. 09, 2022 87 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Age Group (18–35 years old) Tasks Gestures Action (How?) Motivation (Why?) Browse GPS Swipe up, tap Swipe up is used at the lock screen to move the screen upwards and enter the password. The driver also uses swipe up drag the bar upwards and turn on location, the mobile network which is needed to set GPS navigation. The driver uses tap gestures mostly for all steps before setting up the GPS location. Swipe up brings the keyboard to the screen to enter the password. The driver also had to turn on the mobile network and location by swiping up the bar at the home screen. A tap gesture is used to make a selection at each step before setting up a location. Table 2. Most frequently used gestures by age group 36–50 years old and findings of action and motivation level Age Group (36–50 years old) Tasks Gestures Action (How?) Motivation (Why?) Make call Tap, swipe up Most drivers use a tap gesture to select an app, enter a number or alphabet, and select. Swipe up is used at the lock screen to move the screen upwards. Drivers then enter a numerical or alphabetical password or draw pattern recognition. Drivers also swipe up for more applications to be shown on the home screen. Tap gesture is used at most of the steps before performing calling tasks to make various decisions. Swipe up brings the keyboard or dots to the screen. Drivers then enter the password or drag the dots to each other as pattern recognition. Answer call Tap, swipe right, swipe down The driver uses a tap gesture to answer an incoming call from the notification bar. The driver also taps on the speaker or hands-free icon if the driver decides not to hold the phone while driving. Driver swipes right the green icon to answer the incoming call. Driver swipes down the green icon to answer the incoming call as well. Tap gesture is used mainly after answering the call to set the call as a loudspeaker. The driver also taps on the green icon to answer the call and talk on the phone. By swiping the green icon towards the right, the driver allows the incoming calls and is ready to answer. The driver also swipes the green icon downwards to answer the call. Send text message Tap Mainly driver uses tap gesture to select an app, find the sender, type the message, and take other steps before sending the message. Tap gesture is widely used at most steps before performing the messaging task to make various decisions. Read text message Tap The driver uses tap gesture to go message app, select the chat to read the message, sometimes to reply if necessary. Tap gesture is widely used to make a selection of conversation/chat and read the text from the sender. Browse GPS Tap The driver uses tap gestures mostly for all steps before setting up the GPS location. A tap gesture is used to make a selection at each step before setting up a location. Table 1. Most frequently used gestures by age group 18–35 years old and findings of action and motivation level (Continued) 88 http://www.i-jim.org Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… Table 3. Most frequently used gestures by age group more than 50 years old and findings of action and motivation level Age Group (more than 50 years old) Tasks Gestures Action (How?) Motivation (Why?) Make call Tap, swipe up, swipe right The most driver uses a tap gesture to select an app, enter a number or alphabet and make a selection, etc. Swipe up is used at the lock screen to move the screen upwards and enter a password. Swipe right is used at the lock screen to go for the dial pad. Tap gesture is widely used at most of the steps before performing calling tasks to make various decisions. Swipe up brings the keyboard to the screen to enter passwords, it helps drivers to enter passwords easily. Swipe right shortens the steps to go dial pad. The driver can instantly dial the number and make a call without any hustle. Answer call Swipe right, swipe down, tap The driver uses a tap gesture to answer the incoming call from the notification bar. The driver also taps on the speaker or hands-free icon if the driver decides not to hold the phone while driving. Driver swipes right the green icon to answer an incoming call. Driver swipes down the green icon to answer an incoming call. Tap gesture is used mainly after answering the call to set the call as a loudspeaker. The driver also taps on the green icon to answer the call and talk on the phone. By swiping the green icon towards the right, the driver allows the incoming calls and is ready to answer. The driver also swipes the green icon downwards to answer the call. Send text message Tap, swipe up The driver mainly uses tap gestures to select an app, find the sender, type the message, and do other steps before sending the message. Swipe up is used at the lock screen to move the screen upwards and enter the password. Tap gesture is widely used at most steps before performing the messaging task to make various decisions. Swipe up brings the keyboard to the screen to enter the password. Read text message Tap, swipe up The driver uses tap gesture to go message app, select the chat and read the message. Swipe up is used at the lock screen to move the screen upwards and enter a password. Tap gesture is widely used to select conversation/ chat and read the text. Swipe up brings the keyboard to the screen to enter the password. Browse GPS Tap The driver uses tap gestures primarily for all steps before setting up the GPS location. A tap gesture is used to select each step before setting up a location. Thus, Tables 1, 2 and 3 extensively details the most frequently used smartphone ges- tures according to the age group and tasks performed. The table also shows the action and motivation level findings for the gestures. The gestures of age group 18–45 years old show that they are more prone towards a stack of gestures, whereas age group 36–50 years old drivers are more on “tapping” gesture. The drivers of age 50 and above are more towards “swiping” and “tapping” gestures. This concludes that mid and iJIM ‒ Vol. 16, No. 09, 2022 89 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… older drivers are more on simple and easily performed smartphone gestures than the younger drivers. 5 Conclusion & future work Statistically, the number of road crash cases are increasing daily, and more fatalities have been recorded. Even though our government prohibits using a smartphone while driving, the number of drivers using smartphones keeps rising. As the use of smart- phones is so personal, it is almost impossible to eliminate the practice, this research has come out with some frequently used and simple smartphone gestures according to the age group and regular tasks performed. As the current work focuses on getting drivers’ responses with a smartphone while driving, the main aim is to investigate the ideal setup for drivers to interact with smartphone user interfaces. The gestures identified will later assist future smartphone developers who may cater to the need and require- ment in terms of gestural interaction. As future work, the gestures identified will be analyzed further with the Analytical Hierarchy Process (AHP) technique to determine the importance of the gestures identi- fied and which components should be prioritized. This will cope with the current issues of drivers and support them entirely. This will cater to them in any emergence cases where they need to interact with a smartphone, not to encourage them to use it while driving. In the long run, this will reduce the number of car crash cases due to using a smartphone while driving. 6 Acknowledgment This research was fully supported by the Ministry of Higher Education Malaysia, under the Fundamental Research Grant Scheme (FRGS) with Ref. No. FRGS/1/2019/ ICT04/UTP/02/1. The research was conducted at Universiti Teknologi PETRONAS (UTP). 7 References [1] A. Al-Hunaiyyan, R. Alhajri, S. Al-Sharhan, and A. Bimba, “Human-computer interac- tion perspective on mobile learning: Gender and social implications,” Int. J. Interact. Mob. Technol., vol. 15, no. 11, pp. 4–20, 2021. https://doi.org/10.3991/ijim.v15i11.21367 [2] F. M. Caputo, “Gestural interaction in virtual environments: User studies and applications,” CEUR Workshop Proc., vol. 1910, pp. 78–89, 2017. [3] M. I. Qureshi, N. Khan, S. M. Ahmad Hassan Gillani, and H. Raza, “A systematic review of past decade of mobile learning: What we learned and where to go,” Int. J. Interact. Mob. Technol., vol. 14, no. 6, pp. 67–81, 2020. https://doi.org/10.3991/ijim.v14i06.13479 [4] I. Damyanov and N. Tsankov, “Mobile apps in daily learning activities,” Int. J. Interact. Mob. Technol., vol. 12, no. 6, pp. 133–140, 2018. https://doi.org/10.3991/ijim.v12i6.9659 [5] I. Spyropoulou and M. Linardou, “Modelling the effect of mobile phone use on driving behaviour considering different use modes,” J. Adv. Transp., vol. 2019, 2019. https://doi. org/10.1155/2019/2196431 90 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i11.21367 https://doi.org/10.3991/ijim.v14i06.13479 https://doi.org/10.3991/ijim.v12i6.9659 https://doi.org/10.1155/2019/2196431 https://doi.org/10.1155/2019/2196431 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… [6] D. L. Strayer, J. M. Cooper, J. Turrill, J. R. Coleman, and R. J. Hopman, “The smartphone and the driver’s cognitive workload: A comparison of Apple, Google, and Microsoft’s intel- ligent personal assistants,” Can. J. Exp. Psychol., vol. 71, no. 2, pp. 93–110, 2017. https:// doi.org/10.1037/cep0000104 [7] F. H. Burhan, A. Abashah, Z. Ibrahim, and M. Hashim, “Mobile phone distraction fac- tors among Malaysian youth driver,” Int. Conf. Soc. Sci. Innov. Technopreneursh., 2018, [Online]. Available: 335908193_Mobile_Phone_Distraction_Factors_Among_Malaysian_ Youth_Driver [8] D. Rupprecht, D. Künkel, R. Blum, and B. Bomsdorf, “Definition of gesture interactions based on temporal relations,” VISIGRAPP 2018 – Proc. 13th Int. Jt. Conf. Comput. Vision, Imaging Comput. Graph. Theory Appl., vol. 2, no. Visigrapp, pp. 116–123, 2018. https://doi. org/10.5220/0006625201160123 [9] M. Grossman, “Next-Generation User Interface Technologies for Mobile and Consumer Devices,” Report, no. March, p. 71, 2014. [10] S. Lao, X. Heng, G. Zhang, Y. Ling, and P. Wang, “A gestural interaction design model for multi-touch displays,” People Comput. XXIII Celebr. People Technol. – Proc. HCI 2009, pp. 440–446, 2009. https://doi.org/10.14236/ewic/HCI2009.55 [11] Y. Xu and Y. Dai, “Review of hand gesture recognition study and application,” Contemp. Eng. Sci., vol. 10, no. 8, pp. 375–384, 2017. https://doi.org/10.12988/ces.2017.7326 [12] J. O. Wobbrock, A. D. Wilson, and Y. Li, “Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes,” UIST Proc. Annu. ACM Symp. User Inter- face Softaware Technol., pp. 159–168, 2007. https://doi.org/10.1145/1294211.1294238 [13] S. Anwar, S. K. Sinha, S. Vivek, and V. Ashank, “Hand gesture recognition: A survey,” Lect. Notes Electr. Eng., vol. 511, no. 3, pp. 365–371, 2019. https://doi.org/10.1007/978- 981-13-0776-8_33 [14] C. H. Dong, R. G. Ma, D. Zhang, W. T. Zhang, and F. F. Wang, “Research on the influence of smartphone navigation on driving behavior based on real vehicle driving,” Mob. Inf. Syst., vol. 2019, 2019. https://doi.org/10.1155/2019/9527890 [15] D. Basacik, N. Reed, and R. Robbins, “Smartphone use while driving: a simulator study,” Ihs, p. 67, 2011. [16] C. H. Yu and R. C. Miller, “Enhancing mobile browsing and reading,” Conf. Hum. Factors Comput. Syst. – Proc., pp. 1783–1788, 2011. https://doi.org/10.1145/1979742.1979845 [17] R. Augustin, “Just how dangerous is driving and texting,” FMT, Kuala Lumpur, 2017. [18] S. Motamedi and P. Choe, “Cross-cultural design applications in mobile interaction, educa- tion, health, transport and cultural heritage,” vol. 9181, no. August, pp. 0–12, 2015. https:// doi.org/10.1007/978-3-319-20934-0 [19] I. F. Del Amo, E. Galeotti, R. Palmarini, G. Dini, J. Erkoyuncu, and R. Roy, “An innovative user-centred support tool for Augmented Reality maintenance systems design: A prelimi- nary study,” Procedia CIRP, vol. 70, no. May, pp. 362–367, 2018. https://doi.org/10.1016/ j.procir.2018.02.020 [20] J. Ganasan and A. S. Hashim, Gestural Interface to Support Car Drivers Interacting with Smartphone: A Systematic Literature Review, vol. 1294. Springer International Publishing, 2020. https://doi.org/10.1007/978-3-030-63322-6_65 [21] M. Bennett, K. McCarthy, S. O’Modhrain, and B. Smyth, “SimpleFlow: Enhancing gestural interaction with gesture prediction, abbreviation and autocompletion,” Lect. Notes Com- put. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 6946 LNCS, no. PART 1, pp. 591–608, 2011. https://doi.org/10.1007/978-3-642-23774-4_47 [22] H. M. Salman, W. F. Wan Ahmad, and S. Sulaiman, “Usability evaluation of the smartphone user interface in supporting elderly users from experts’ perspective,” IEEE Access, vol. 6, no. April, pp. 22578–22591, 2018. https://doi.org/10.1109/ACCESS.2018.2827358 iJIM ‒ Vol. 16, No. 09, 2022 91 https://doi.org/10.1037/cep0000104 https://doi.org/10.1037/cep0000104 http://335908193_Mobile_Phone_Distraction_Factors_Among_Malaysian_Youth_Driver http://335908193_Mobile_Phone_Distraction_Factors_Among_Malaysian_Youth_Driver https://doi.org/10.5220/0006625201160123 https://doi.org/10.5220/0006625201160123 https://doi.org/10.14236/ewic/HCI2009.55 https://doi.org/10.12988/ces.2017.7326 https://doi.org/10.1145/1294211.1294238 https://doi.org/10.1007/978-981-13-0776-8_33 https://doi.org/10.1007/978-981-13-0776-8_33 https://doi.org/10.1155/2019/9527890 https://doi.org/10.1145/1979742.1979845 https://doi.org/10.1007/978-3-319-20934-0 https://doi.org/10.1007/978-3-319-20934-0 https://doi.org/10.1016/j.procir.2018.02.020 https://doi.org/10.1016/j.procir.2018.02.020 https://doi.org/10.1007/978-3-030-63322-6_65 https://doi.org/10.1007/978-3-642-23774-4_47 https://doi.org/10.1109/ACCESS.2018.2827358 Paper—Task Based Test Case Generation on Available Gestural Interaction of Smartphone for Improved… [23] Z. Chen, et al., “User-defined gestures for gestural interaction: Extending from hands to other body parts,” Int. J. Hum. Comput. Interact., vol. 34, no. 3, pp. 238–250, 2018. https:// doi.org/10.1080/10447318.2017.1342943 [24] K. A. M. Isa, et al., “Mobile phone usage behaviour while driving among educated young adults in the urban university,” Procedia – Soc. Behav. Sci., vol. 36, no. June 2011, pp. 414–420, 2012. https://doi.org/10.1016/j.sbspro.2012.03.045 [25] M. F. M. Siam, M. H. M. Isa, N. Borhan, A. Sukardi, and W. S. Voon, “Measurement of driver distraction in Malaysia’s traffic environment: A driving simulator study,” J. Mech. Eng. Sci., vol. 8, no. June, pp. 1472–1480, 2015. https://doi.org/10.15282/jmes.8.2015.21.0143 [26] K. Tippey, E. Sivaraj, and T. K. Ferris, “Driving while interacting with google glass: Inves- tigating the combined effect of head-up display and hands-free input on driving safety and multitask performance,” Hum. Factors J. Hum. Factors Ergon. Soc., vol. 59, no. 1, 2017. https://doi.org/10.1177/0018720817691406 [27] L. P. De Oliveira, M. A. Wehrmeister, and A. S. De Oliveira, “Systematic literature review on automotive diagnostics,” Brazilian Symp. Comput. Syst. Eng. SBESC, vol. 2017, pp. 1–8, 2017. https://doi.org/10.1109/SBESC.2017.7 [28] S. A. Birrell and M. Fowkes, “Glance behaviours when using an in-vehicle smart driving aid: A real-world, on-road driving study,” Transp. Res. Part F Traffic Psychol. Behav., vol. 22, pp. 113–125, 2014. https://doi.org/10.1016/j.trf.2013.11.003 [29] F. Guo, et al., “The effects of age on crash risk associated with driver distraction,” Int. J. Epidemiol., vol. 46, no. 1, pp. 258–265, 2017. https://doi.org/10.1093/ije/dyw234 [30] J. He, J. S. McCarley, K. Crager, M. Jadliwala, L. Hua, and S. Huang, “Does wearable device bring distraction closer to drivers? Comparing smartphones and Google Glass,” Appl. Ergon., vol. 70, no. February 2017, pp. 156–166, 2018. https://doi.org/10.1016/ j.apergo.2018.02.022 8 Authors Jayasankari Ganasan is currently pursuing Master of Science (MSc) in Information Technology at Universiti Teknologi PETRONAS (UTP), Malaysia. She completed her Foundation and Bachelor of Technology (B.Tech) degrees from Universiti Teknologi PETRONAS (UTP) in 2019. Her research interest includes human-computer interac- tion, user experience, and usability. (Email: jayasankari_20000248@utp.edu.my) Ahmad Sobri Hashim holds a post senior lecturer at Computer and Information Sciences Department (CISD), Universiti Teknologi PETRONAS (UTP), Malaysia. He obtained his Doctor of Philosophy (PhD), Master of Science (MSc) and Bache- lor of Technology (B.Tech) degrees from UTP in the Information Technology field. He has been teaching since 2014, and he is very passionate about teaching program- ming courses, including Structured Programming and Web Programming. His research interests include IT applications, E-Systems, human-computer interaction, educational technologies, learning disabilities and information systems. (Email: sobri.hashim@utp. edu.my) Article submitted 2022-02-16. Resubmitted 2022-03-28. Final acceptance 2022-03-28. Final version published as submitted by the authors. 92 http://www.i-jim.org https://doi.org/10.1080/10447318.2017.1342943 https://doi.org/10.1080/10447318.2017.1342943 https://doi.org/10.1016/j.sbspro.2012.03.045 https://doi.org/10.15282/jmes.8.2015.21.0143 https://doi.org/10.1177/0018720817691406 https://doi.org/10.1109/SBESC.2017.7 https://doi.org/10.1016/j.trf.2013.11.003 https://doi.org/10.1093/ije/dyw234 https://doi.org/10.1016/j.apergo.2018.02.022 https://doi.org/10.1016/j.apergo.2018.02.022 mailto:jayasankari_20000248@utp.edu.my mailto:sobri.hashim@utp.edu.my mailto:sobri.hashim@utp.edu.my