International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 13, 2021 Paper—The Design and Evaluation of a New Smartwatch User Interface The Design and Evaluation of a New Smartwatch User Interface https://doi.org/10.3991/ijim.v15i13.22701 Karan Singh Chaudhary, Pietro Murano () OsloMet – Oslo Metropolitan University, Oslo, Norway pietro.murano@oslomet.no Abstract—This paper discusses and presents a new prototype design for a smartwatch user interface. The user interface was designed aiming to adhere to some of the main universal design principles and be a more usable design. The prototype user interface was then compared with a Samsung Gear S3 smart- watch user interface. The comparison was done via an experiment and a series of realistic tasks. The aspects being investigated were task times, errors and subjective user satisfaction. The data collected were statistically analysed. The overall results showed that the prototype user interface fostered faster task times, fewer errors and more user satisfaction. Keywords—Smartwatches, User Interfaces, Interaction, Usability, Universal design, prototype 1 Introduction The present-day smartwatch and other wearables have a similar physical appear- ance as traditional watches. The smartwatch and other wearables are revolutionized versions of traditional watches equipped with smart interfaces and complex features. The first computing in wearables began in 1955 and was developed by Edward O. Thorp in 1961. This was a ‘cigarette pack sized’ device to analyze the motion of rou- lette wheels [27]. This indicated the potential of wearables resulting in time with the introduction of the first smartwatch. This was the IBM Linux Watch launched in 2000 [19]. Many researchers have concentrated their efforts on producing novel gesture recognition for smartwatches and ways of expanding the small display of a smart- watch, with the aim of making the whole interaction and ownership experience of a smartwatch better (see Section 2 for more details). Some researchers have also concentrated their efforts on using smartwatches in a very specialized context. For example, the work in [1] investigated the option of using a smartwatch for indoor localization, linked to care of the elderly. However, to our knowledge there have been fewer efforts aiming to produce a uni- versally designed and usable user interface by means of careful, more thoughtful and 128 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i13.22701 mailto:example@example.org Paper—The Design and Evaluation of a New Smartwatch User Interface rigorously evaluated designs whilst working within the limits of the screen real estate of smartwatches. In this paper we present a new universally designed and more usable smartwatch user interface. We will show with empirical evidence and statistically significant results, that certain changes to the current design used in a Samsung Gear S3 smart- watch can result in important and useful usability gains. These are achieved purely by careful design and not different gestures or attempts to increase the screen size. The rest of this paper is structured as follows: Section 2 will present some relevant literature. Section 3 will present the new designs. Section 4 will describe an empirical experiment along with the statistical analysis and Section 5 will conclude with a dis- cussion and conclusions. 2 Selected Literature Review This section discusses some of the previous work that has been done concerning smartwatches, small screens and wearable devices. The selected literature will specif- ically not cover in detail the hardware development aspects as this is considered by the authors to be unnecessary in the context of our work. We therefore concentrate more on the interaction and user interface aspects of smart watches. 2.1 Overview of smartwatch interfaces Smartwatches are categorized as mini versions of computers whose primary func- tion is to collect data and present relevant information, such as notifications and emails [4]. Smartwatches are recognized as a device which is usually worn on the wrist by the user and used to get information conveniently on the wrist and stay con- nected to the things that matter through synchronization with other devices like smartphones [3]. Smartwatch use has led to certain characteristics of use. One exam- ple concerns a survey in [18] where it found that users use the smartwatch more often during lunchtime or when they cannot or are unable to use a smartphone. Similarly, it was found that 40% of women and 30% of men were likely to miss phone calls due to the way they carry the mobile phone which suggests that the mobile phone may not be easily accessible for them despite having it with them [6]. Similarly, wearables like smartwatches have been quicker to be accessed than mobile devices [2]. These as- pects can give an indication concerning which features can be included in a smart- watch interface to enable users to easily access information through a smartwatch without accessing the smartphone. In the context of wearable systems, the work in [24] suggested that it is important for interfaces to be aware of the user’s context so that it can recognize and respond to the users’ cognitive need and anticipate as required. They further concluded that the interface should be able to maintain the user’s attention and not cause information overload. A study in the context of tasks involving geographical maps, comparing the visual- ization techniques between various types of display found that among them, small iJIM ‒ Vol. 15, No. 13, 2021 129 Paper—The Design and Evaluation of a New Smartwatch User Interface screen devices performed worst in comparison to other bigger and medium sized displays [11]. They used the same content to be displayed on various displays. Some of the results suggest that content at the interface should be designed differently with respect to display size. The work in [21] also suggests that display size matters while designing interactions for various types of displays as interaction zones vary as per display size rather than visualization type. Another study done to find out why and how people use smartwatches in their dai- ly life, found that smartwatches are able to reduce the dependency on the mobile phone due to the usefulness of availability of notifications in social situations [3]. They also concluded that although smartwatches are useful at specific times, they would require additional functionality if the overall user base was to increase. This suggests that there are still more improvements to be done in smartwatch interfaces to create a rich content smartwatch. 2.2 Barriers and usability issues in smartwatch interfaces Although small screen devices such as smartwatches have gained popularity and are expected to increase in their proliferation, smartwatches still have some limita- tions which hold them back in terms of usability and user satisfaction. The design of an interface is linked to the size of the device which means the con- tent of the interface should be designed in relation to the screen size. The screen size matters when it comes to interaction efficiency and effectiveness both in terms of accessibility and usability [22]. Smartwatches are usually worn on a wrist thus inter- action is usually done using only one hand. One hand interaction can encounter some limitations and problems due to limited thumb reach, the fat-finger problem as well as when the other hand is occupied or not available [29]. With the aim of overcoming the barriers of small screen display size of smart- watches, various additional efforts and approaches of interaction methods have been investigated. Some examples include, the use of gesture-based input using the wrist as a joystick [7], motion gestures [17], performing tasks through touch-free interaction, solving the fat-finger problem [12], [15], using a camera in a smartwatch to capture finger gestures to use as input [28], use of multi-touch gestures such as pinch, per- formed on the back of a hand with mid-air gestures [25], gaze interaction to give commands to the smartwatch for tasks [8] and use of the user’s skin for providing a larger display [9], [10]. A study of smartwatch usage and usability issues involving 30 participants from 20 to 43 years old, found the following usability issues [5]: • Text font and icons were small in size and easily blocked by fingers while interact- ing with the smartwatch interface. • It was difficult to read content immediately due to the small screen size and it be- came more difficult when the content was long. • Information shown on the smartwatch interface display did not provide enough or sufficient details. 130 http://www.i-jim.org Paper—The Design and Evaluation of a New Smartwatch User Interface • It was difficult to select the targeted object or icons accurately and perform touch gestures like tapping and swiping. • Excessive scrolling was required to find the target from a long list, resulting in lack of user control and need of repetitive interaction to perform the task. • The interaction was not responsive enough to touch input, especially in users’ real situations, such as walking. • Difficult to learn and required initial learning to be familiar with the smartwatch. • Due to different and complex UI design and content display layouts, there were problems in performing tasks easily. As stated in [5], their study found that the users wanted larger displays in the smartwatch and the accuracy of input interaction to be improved. Further, users pre- ferred to check the weather in the smartwatch. Also, they found that usability of small screen devices such as smartwatches were poor in terms of interacting with visually rich content. Similarly, their study also found that users were not satisfied with the current interface design of the smartwatch and expected new smartwatch interfaces to reduce usability issues. Touch input is considered as a general or primary input method for smartwatches, but there are various alternative gestures related to the input approach being re- searched and developed over time. When the various input methods were compared, it was found that touch input modality was faster than other input methods even for linear list search [13]. Ref. [23] indicated that smartwatches have major constraints of having small screens which creates a challenge in defining new instances of input and output capabilities. They further stated that although research has been done, it does not seem to address the constraints of smartwatches properly or in a realistic way. Therefore, they concluded that the screen restrictions of smartwatches will require user interface designers to think of new interaction techniques and designs. Ref. [20] considers that any user interface needs these following three features to fulfill the requirement of a usability approach: • The interface should be easy to learn and use by the users. • The interface should be able to enable the users to remember the patterns of opera- tional methods of the interface after some time of non-use. • The design of the interface should help the users to minimize making errors while using it and should easily resolve the errors if it is made. To create usable smartwatch interfaces, in [5] they propose usability principles that can be applied while designing smartwatch interfaces. They proposed several catego- ries in their principles. Their ‘Information Display’ and ‘Learnability’ categories are similar to other general user interface guidelines concerning these issues. Their ‘Con- trol’ category is more specific to smartwatches and contains aspects to do with being able to interact well using a fingertip etc. Their ‘Interoperability’ category deals most- ly with connectivity issues between a smartwatch and a telephone. It also deals with some aspects of the user interface. Their ‘Preference’ category deals with subjective issues of a user’s experience with the smartwatch. iJIM ‒ Vol. 15, No. 13, 2021 131 Paper—The Design and Evaluation of a New Smartwatch User Interface The usability issues of smartwatch interfaces can be divided into two groups, i.e. hardware and software. A study conducted on the usability issues showed that screen- size, device, size, resolution, and battery are related to the hardware of a smartwatch, while typography, navigation, button location, iconography, and interaction technique are related to the software of a smartwatch interface [14]. In Ref. [14] it was found that font size was important for older adults and button location should be appropriate with use of proper icon sizes and design. Similarly, interaction techniques should be represented in a simple and effective way for better usability [14]. As this selected literature review has shown, there have been research efforts over a number of years concerning smartwatches, small screens in general and wearable devices. The majority of these studies indicate that the design of smartwatches and particularly their user interfaces need further work if the smartwatch is to become a truly useful and user-friendly device. In line with what has been observed from previ- ous works we will in the next sections describe a new user interface and the results of an evaluation of the new user interface. 3 New User Interface Design Prototype Aiming to make improvements to smartwatch user interfaces we began by develop- ing a prototype user interface for the Samsung Gear S3. This was then compared in an empirical experiment with the default Samsung Gear S3 user interface. The new user interface was developed by being guided by the shortcomings found in the previous work done in this area (see previous main section). The new user inter- face was also developed in line with the Universal Design principles [26] applied to user interfaces and we note that in [5] their design guidelines have some overlap with the universal design principles. Aiming for a fair comparison, four popular applications were used in both smart- watch interfaces: Message, Contacts, Call, and Weather. Each application consisted of a series of screens which included a series of touch interactions for navigation. A screenshot of the new universally designed smartwatch interface prototype with its application is shown below. 132 http://www.i-jim.org Paper—The Design and Evaluation of a New Smartwatch User Interface Fig. 1. Universally designed smartwatch interface prototype design chart Figure 1, Screen 1 shows the main user interface displaying the time and date to the user. Furthermore, the text ‘double tap to begin...’ is written to guide the user to double tap the screen to interact with the screen to go to the next page. After the user double taps anywhere on the screen four applications named as ‘Message’, ‘Contacts’, ‘Call’, and ‘Weather’ become available to the user (Figure 1, Screens 2-5). All the applications consist of an icon and text indicating what type of app it is. Initially, the ‘message’ app is shown (Figure 1, Screen 2). In the ‘message’ app option, an icon of a message is used followed by the text ‘message’ just below the icon. This helps the user to identify the type of option that can be selected. The other menu options can be retrieved by the user swiping up or down with touch gestures. Similarly, for a back or previous option, the back button of the smartwatch by default can be used which is situated on the top right side of the smartwatch (image not in- cluded). To help the user know the navigation direction through various apps, small arrows are shown at the top and bottom of the screen (See Figure 1, Screens 2-5). To view a message, the user can touch the ‘Message’ option (Figure 1, Screen 2) and go to the message section where the user can view the message of a contact per- son (Figure 1, Screen 6). The back option can be generally achieved by pressing the back button of the smartwatch but nevertheless, the option to go back is also provided at the bottom of the message section of the contact (Figure 1, Screen 10). To go back to the main screen, the user can simply press the back button and reach the destina- tion. Similarly, the ‘Contacts’ option provides the detailed information of the contact person, including name, contact number and an icon showing the contact’s gender iJIM ‒ Vol. 15, No. 13, 2021 133 Paper—The Design and Evaluation of a New Smartwatch User Interface (Figure 1, Screen 7). Further, two options can be seen (Figure 1, Screen 7) which enable a user to either call or message the respective contact. Once the call option is tapped, a screen is shown calling the relevant person with an option to end the call if required (Figure 1, Screen 11). The messaging option takes the user to the message section of the ‘Message’ option of the relevant contact. This shows the interconnec- tion between options. The screen displays the contact information of the first contact at first. Then the user can use swipe up or swipe down to access the other contacts as required. Another option is called ‘Call’ (Figure 1, Screens 4, 8 and 11). This can be used to place a call by dialing a number. When the user taps the ‘Call’ option (Figure 1, Screen 4), a screen opens showing the number system in grid format with a delete button at the top right side and a button to place a call after dialing a number (Figure 1, Screen 8). The button to place a call is shown in green as seen in most other inter- active systems with text such as ‘call’ written with the icon representing the call func- tion. Once the user presses the desired number and presses the ‘call’ button, a screen is displayed showing that the call is being placed (Figure 1, Screen 11). Further, the user can end the call by pressing the end call icon shown at the bottom of the screen (Figure 1, Screen 11). The last option is called ‘Weather’ which is initialized to present the weather in- formation to the user (Figure 1, Screens 5 and 9). When the user taps the ‘weather’ option, a screen opens (Figure 1, Screen 9) showing weather information for a loca- tion at the top of the screen, temperature on the left and time on the right. There is also an icon to represent the weather condition (e.g. a sun etc.) as well as an equiva- lent short text label relevant to that weather condition. Furthermore, the day of the week and date is shown at the bottom of the screen. This section includes information for the week which can be visualized by swiping left or right. To help the user to recognize the appropriate gestural interaction, a small arrow is presented to the left and right side of the screen indicating left and right swipe for interaction between various content sections. The design features that are detailed above and viewable in Figure 1 were aiming to adhere to universal design principles. Universal Design Principle Three: Simple and Intuitive Use [26] states that the design should be understandable irrespective of who the user is. In our design above we have included clear labelling on all icons, rather than relying on users recognizing and/or remembering all icons. We have also given guidance on the direction of swiping, as many mobile-based apps can hide such visual cues causing designs to be unintuitive. The initial screen (Figure 1, Screen 1) also gives guidance regarding how to proceed next. These are all features that con- tribute to a user interface that should be simple to use and intuitive. These design aspects also link with Universal Design Principle Four: Perceptible Information [26]. This principle is about giving information to the users that is needed and is effective. As stated above the design of our prototype aims to achieve this as described. The design does not rely purely on iconic representations, but these are all labelled appro- priately. Furthermore, since most smartwatches have a high degree of customization available concerning aspects such as colours, fonts and brightness levels these there- fore feed into achieving Principle Four. Our designs further link with Universal De- 134 http://www.i-jim.org Paper—The Design and Evaluation of a New Smartwatch User Interface sign Principle Five: Tolerance for Error [26]. In part, this principle concerns how various elements are arranged. In this case the elements concern the various screens and interaction steps within the apps. This has been achieved by trying to have clear and simple linked elements and where potential for error exists, we have tried to make it easy to recover from this. One example concerns the possibility of a user dialing a number by mistake. The screen that performs the dialing operation has a clear ‘end’ function. The relevant screens in addition have small arrows to show the user the correct swipe directions thus avoiding swiping in the wrong direction. In the next section we present the details of an experiment that was carried out to evaluate if our aims had been met concerning an improved smartwatch user interface. 4 Experimental Evaluation of the New User Interface 4.1 Participants In this experiment, a total of 30 participants were included. The participants were university students studying at the master level and some were skilled workers (appli- cation developer, designer). The participants were approached with email invitations for participating in the experiment. When they accepted the invitation, then the loca- tion of the experiment was given as a choice between the participant’s own place or university project room. The experiment was conducted with the presence of the ex- perimenter and participant only. 4.2 Experimental design The experiment for this investigation was conducted using a within users design. This in practice meant the participants were able to experience both user interfaces and in turn provide useful opinions on each interface from a perspective of having experienced both user interfaces. 4.3 Variables The independent variables were the two smartwatch interfaces (Samsung Gear S3, new prototype smartwatch interface). The independent variables consisted of four tasks each which were designed to navigate through four different applications of the interface. The defined dependent variables were performance and user satisfaction. The de- pendent measures for performance were task completion time and number of errors. Errors were defined in two ways. The first one included a user tapping the incorrect option. The second one included a user swiping in the wrong direction. All the errors were recorded combining all the errors into a single score for statistical analysis. The dependent measures for user satisfaction involved the use of a designed post- experiment questionnaire. The questionnaire was designed to elicit the opinions of the participants in terms usability, interface design, task design and overall experience iJIM ‒ Vol. 15, No. 13, 2021 135 Paper—The Design and Evaluation of a New Smartwatch User Interface while using both smartwatch user interfaces. The questionnaire was designed around participants rating their opinions on a Likert-type [16] scale. The scale used was from 1 to 7, where in each question a 7 score was the highest positive score available. 4.4 Apparatus and materials For carrying out the experiment and completing the data collection, the following systems and materials were used: • Samsung Gear S3 with 1.3” circular screen size with (360x360 resolution), Exynos 7270 processor with 768MB RAM • Stopwatch • Participant Information Sheet • Consent form • Pre-Experiment Questionnaire • Tasks information for participants • Post-Experiment Questionnaire 4.5 Task design A total of four tasks were designed to test four applications (see Figure 1) of the smartwatch interface as well as interaction styles used in the smartwatch interface. The main aim of the tasks was to evaluate how users intend to interact with the inter- face while searching for specific information or completing certain tasks. Further- more, this research study aims to compare two smartwatch interfaces determining which interface is more usable. The four tasks are shown in Tables 1 and 2 for each smartwatch interface. Table 1. Tasks to be performed in Samsung Gear S3 interface Task ID Task Details A Open “Message” application and find the message of “MyCall”. B Open “Contact” app and call the contact named “Amrit Sogn Oslo”. C Open “Call” app and enter the number “48664579” and press “call” button. D Open “Weather” app and find the weather condition in “Wednesday” Table 2. Tasks to be performed in New Prototype Smartwatch Interface Task ID Task Details A Open “Message” app and find the message of “Diana”. B Open “Contact” app and call the contact named “Carol”. C Open “Call” app and enter the number “48664579” and call button. D Open “Weather” app and find the weather condition in “Wednesday”. Randomization was used to minimize possible learning effects from the partici- pants during the tasks. Therefore, the user interfaces and the tasks were randomized in 136 http://www.i-jim.org Paper—The Design and Evaluation of a New Smartwatch User Interface their order of presentation to each participant. A snippet of the randomization under- taken is presented in Table 3. Table 3. Snippet of Randomization for Experimental Groups and Tasks User ID Interface Task Group Interface Task Group Interface Task Group Interface Task Group 1 S3 A Prototype B S3 C Prototype D … … … … … … … … … 30 Prototype D S3 A Prototype B S3 C S3 = Samsung Gear S3 smartwatch interface, Prototype = Prototype Smartwatch interface; A = Task A, B = Task B, C = Task C, D = Task D 4.6 Procedure Before starting the experiment, a pilot test was performed by two students of Oslo Metropolitan University pursuing a Master level degree. The main goal of the pilot testing was to ensure the real experiment could be conducted without any bias and issues. The pilot test participants were provided tasks to perform between the two smartwatch interfaces. After the pilot test two main issues were discovered. • It was observed that the pilot test participants were able to learn the pattern of tasks through previous tasks, thus potentially creating bias. Therefore, randomization as described in Table 3 was adopted. • During the design of the experiment, it was assumed that the participants may re- quire training before the real experiment. However, during the pilot test, it was found that the participants were able to use the smartwatch interface easily. There- fore, the idea of initial training was removed from the experiment. The actual experiment procedure was started after the participants were given de- tails about the research study and experiment. The experiment was divided into three sections: pre-experiment, experiment execution and post-experiment. The whole ex- periment conformed with Norwegian ethical requirements for research of this kind. The pre-experiment section involved welcoming the participants and then giving some explanation about the experiment. The participants were provided with partici- pant information sheets to provide detailed information about the experiment and a consent form to provide details about ethical issues and confidentiality. Upon comple- tion of reading this material, participants read and signed the consent form. Partici- pants were then given a pre-experiment questionnaire to complete with the aim of gathering non-sensitive background information about the participants. The experiment was carried out on an actual Samsung Gear S3 smartwatch using the tasks described in Tables 2 and 3. The participants were given the choice to wear the smartwatch on any wrist they wanted and in their preferred physical position. A stopwatch was used to record the time taken and any errors made were recorded man- ually. After the experiment was completed, the participants were given a post-experiment questionnaire to complete. iJIM ‒ Vol. 15, No. 13, 2021 137 Paper—The Design and Evaluation of a New Smartwatch User Interface 4.7 Results The data collected from the experiment was analysed statistically through a one- way repeated MANOVA test. This was with the aim of observing if there would be statistically significant differences between the two user interfaces for performance and user satisfaction. Performance involved task times and errors. User satisfaction was based on the responses given by the participants via the post-experiment ques- tionnaire. The testing involved the correlation relationship check, normal distribution test, and one-way MANOVA test. All the data collected passed the correlation rela- tionship check and normal distribution test. To keep the paper as brief as possible, only statistically significant results are discussed in this paper. Total time taken for task times, Table 4 presents the means and standard devia- tions. In all cases the prototype user interface fostered faster task times than the Sam- sung Gear S3 user interface. Table 4. Means and Standard Deviations for Task Times Means and Standard Deviations – Task Times Interface Mean Std. Deviation N Task 1 Prototype 14.00 2.913 30 Samsung 16.40 2.931 30 Task 2 Prototype 14.33 1.213 30 Samsung 16.37 1.956 30 Task 3 Prototype 15.00 1.819 30 Samsung 17.20 2.369 30 Task 4 Prototype 14.90 1.845 30 Samsung 16.60 2.401 30 The one-way MANOVA test for total time taken shows that there were significant differences between the two smartwatch interfaces for Task 1 (F (1, 58) = 10.120; p < 0.05; ƞp2= 0.15), Task 2 (F (1, 58) = 23.413; p < 0.05; ƞp2 = 0.29), Task 3 (F(1, 58) = 16.270; p < 0.05; ƞp2 = 0.22), and Task 4 (F(1, 58) = 9.456; p < 0.05; ƞp2 = 0.14). Clearly all four tasks were completed significantly quicker when carried out with the prototype user interface. Total errors made for errors, Table 5 presents the means and standard deviations. In all cases the prototype user interface fostered fewer errors than the Samsung Gear S3 user interface. 138 http://www.i-jim.org Paper—The Design and Evaluation of a New Smartwatch User Interface Table 5. Means and Standard Deviations for Errors Means and Standard Deviations - Errors Interface Mean Std. Deviation N Task 1 Prototype .47 .629 30 Samsung .80 .714 30 Task 2 Prototype .33 .606 30 Samsung .80 .887 30 Task 3 Prototype .27 .450 30 Samsung .67 .844 30 Task 4 Prototype .23 .504 30 Samsung .73 .868 30 The one-way MANOVA test shows that there was not a significant difference in terms of total errors made during the completion of Task 1. However, there were significant differences in terms of errors made during the completion of Task 2 (F (1, 58) = 5.661; p < 0.05; ƞp2 = 0.089), Task 3 (F (1, 58) = 5.246; p < 0.05; ƞp2 = 0.083) and Task 4 (F (1, 58) = 7.440; p < 0.05; ƞp2 = 0.0.114). User satisfaction data analysis: There were a total of 15 post-experiment ques- tions in the post-experiment questionnaire covering aspects to do with the user inter- face, the process of using the user interface and participant emotions throughout the interaction. Furthermore, the post-experiment questionnaire was used for collecting the participant opinions for both the Samsung Gear S3 smartwatch interface and the New Prototype smartwatch interface. The questionnaire elicited responses on a Lik- ert-type scale (1-7) where 1 indicated the most negative response possible and 7 indi- cated the most positive response possible. Concerning the questions about pleasantness of the user interface, the content be- ing read with ease, the tidiness of the content, ease of performing the tasks, ease of learning the user interface, and stressfulness in using the user interface, no statistically significant results were observed. The fourth question concerned whether the font size was of the right size for the participants. Overall, the font size in the prototype smartwatch interface (mean = 4.90; SD = 1.242) was scored more positively than the Samsung Gear S3 smartwatch inter- face (mean = 3.93; SD = 1.230). The difference was statistically significant, (F (1, 58) = 9.179; p < 0.05; ƞp2 = 0.137). The fifth question concerned color contrast levels at the user interface. Overall, the prototype smartwatch interface (mean = 4.77; SD = 1.382) was scored more positively than the Samsung Smartwatch interface (mean = 3.97; SD = 1.217). The difference was statistically significant, (F (1, 58) = 5.662, p<0.05; ƞp2 = 0.089). The sixth question concerned the overall ease of use of the user interface. Overall, the prototype smartwatch interface (mean = 5.07; SD = 1.258) was scored more posi- tively than the Samsung Gear S3 smartwatch interface (mean = 4.07; SD = 1.258). The difference was statistically significant, (F (1, 58) = 9.484; p< 0.05; ƞp2 = 0.141). The seventh question concerned opinions about the icon size being sufficient to en- able easy tapping and/or selection. Overall, the prototype smartwatch interface (mean = 5.00; SD = 1.259) was scored more positively than the Samsung Gear S3 interface iJIM ‒ Vol. 15, No. 13, 2021 139 Paper—The Design and Evaluation of a New Smartwatch User Interface (mean = 3.60; SD = 1.163). The difference was statistically significant, (F (1, 58) = 20.014; p < 0.05; ƞp2 = 0.257). The eighth question concerned ease of navigation through the apps from the menu. Overall, the prototype smartwatch interface (mean = 5.00; SD = 1.259) was scored more positively than the Samsung Gear S3 smartwatch interface (mean = 4.00; SD = 1.259). The difference was statistically significant, (F (1, 58) = 9.457; p< 0.05; ƞp 2 = 0.140). The ninth question concerned the ease of viewing and understanding the content of the interface. Overall, the prototype interface (mean = 5.07; SD = 1.258) was scored more positively that the Samsung Gear S3 smartwatch interface (mean = 3.80; SD = 1.349). The difference was statistically significant, (F (1,58) = 14.147; p < 0.05; ƞp2 = 0.196). The twelfth question concerned the ease of interaction with the interface as it was currently designed. Overall, the prototype interface (mean = 5.07; SD = 1.258) was scored more positively than the Samsung Gear S3 smartwatch interface (mean = 4.07; SD = 1.258). The difference was statistically significant, (F (1, 58) = 9.484; p < 0.05; ƞp2 = 0.141). The thirteenth question concerned participants’ beliefs regarding whether the user interface would be easy to remember after some time of non-use. Overall, the proto- type interface (mean = 4.97; SD = 1.299) was scored more positively than the Sam- sung Gear S3 smartwatch interface (mean = 3.80; SD = 1.349). The difference was statistically significant, (F (1, 58) = 11.636; p< 0.05; ƞp2 = 0.167). The fifteenth question concerned participants’ frustration levels while using the us- er interface. Overall, the prototype interface (mean = 5.07; SD = 1.258) was scored more positively than the Samsung Gear S3 interface (mean = 4.07; SD = 1.345). The difference was statistically significant, (F (1, 58) = 9.484; p < 0.05; ƞp2 = 0.141). This section has presented the statistically significant results from the data collect- ed from the experiment. The performance data collected was statistically significant showing the prototype user interface fostering faster task times and fewer errors. The subjective opinions data from the post-experiment questionnaire was also in most cases statistically significant, showing the prototype user interface fostering more positive opinions. In the next section we will discuss the results and present some conclusions. 5 Discussion and Conclusion This study aimed to find out whether the smartwatch interface if designed and de- veloped using careful design and universal design principles can provide better usabil- ity and user satisfaction. In this study, a Samsung Gear S3 smartwatch interface was selected and compared with a newly developed universally designed smartwatch in- terface prototype. The evaluation results in the previous section show categorically statistically sig- nificant results for performance aspects concerning task times and errors. The proto- type user interface fostered faster times and fewer errors. There were also categorical- 140 http://www.i-jim.org Paper—The Design and Evaluation of a New Smartwatch User Interface ly statistically significant results for subjective satisfaction aspects. Nine out of the 15 questions suggested participants were significantly more satisfied with the experience of using the prototype user interface. The other six questions were non-significant in nature. However, the scores given by the participants in these six questions were all more positive towards the prototype user interface when compared with the Samsung Gear S3 user interface scores. The actual Samsung Gear S3 user interface achieved no statistically significant results across any of the factors measured in the experiment. These results stand out to suggest clearly that employing careful design, appropri- ate human computer interaction (HCI) techniques and universal design principles for smartwatch user interfaces will lead to a better-quality user interface and better user experience. We specifically concentrated on universal design principles three, four and five in the design effort. In line with these universal design principles, we de- signed the new smartwatch user interface to avoid ambiguity, guide the user’s interac- tion flow and where there could be the possibility of some error happening, clear visible ways to recover. The results were categorical with the prototype user interface outperforming the Samsung Gear S3 user interface. Our work makes some significant contributions to the field of HCI. Firstly, while investigating new gestures and methods for expanding screen real estate are interest- ing and should continue, our work suggests that good usability and user experience can be achieved without these specific areas of investigation. This should be of inter- est and of value to any corporation producing such products as it means they can make large improvements to their products by simply being more careful in how they design their smartwatch user interfaces. The second significant contribution we sug- gest is that to our knowledge no one has designed, evaluated and documented a new smartwatch user interface design where the design was strongly guided by the univer- sal design principles. In future work, we would like to have larger participant numbers for an experi- ment. The study provides some variation in the age groups only so we would like to add more participants with various impairments to achieve wider reaching conclu- sions. The interface designed in this study is simple with only four applications, we would therefore like to add more complexity and add more applications for testing and evaluation. Similarly, in this study only touch interaction style was used, in future other forms of interaction methods and various feedback methods can be applied and evaluated. Lastly, longer term studies could give further information regarding the user interface design. 6 References [1] Árvai, L. (2021) Application of Smartwatches in Elderly Care with Indoor Localization Func- tionality, International Journal of Interactive Mobile Technologies, Vol 15, No 5. [2] Ashbrook, D. L., Clawson, J. R., Lyons, K., Starner, T. E., & Patel, N. (2008). Quickdraw: the impact of mobility and on-body placement on device access time. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/1357054.1357092 iJIM ‒ Vol. 15, No. 13, 2021 141 https://doi.org/10.1145/1357054.1357092 Paper—The Design and Evaluation of a New Smartwatch User Interface [3] Cecchinato, M. E., Cox, A. L., & Bird, J. (2015). Smartwatches: the Good, the Bad and the Ugly? Paper presented at the Proceedings of the 33rd Annual ACM Conference extended ab- stracts on human factors in computing systems. https://doi.org/10.1145/2702613.2732837 [4] Chuah, S. H.-W., Rauschnabel, P. A., Krey, N., Nguyen, B., Ramayah, T., & Lade, S. (2016). Wearable technologies: The role of usefulness and visibility in smart-watch adoption. Com- puters in Human Behavior, 65, 276-284. https://doi.org/10.1016/j.chb. 2016.07.047 [5] Chun, J., Dey, A., Lee, K., & Kim, S. (2018). A qualitative study of smartwatch usage and its usability. Human Factors and Ergonomics in Manufacturing & Service Industries, 28(4), 186- 199. [6] Cui, Y., Chipchase, J., & Ichikawa, F. (2007). A cross culture study on phone carrying and physical personalization. Paper presented at the International Conference on Usability and Internationalization. https://doi.org/10.1007/978-3-540-73287-7_57 [7] Gong, J., Yang, X.-D., & Irani, P. (2016). WristWhirl: One-handed Continuous Smartwatch Input using Wrist Gestures. Paper presented at the Proceedings of the 29th Annual Symposi- um on User Interface Software and Technology, Tokyo, Japan. https://doi.org/10.1145/ 2984511.2984563 [8] Hansen, J.P., Biermann, F., Madsen, J.A., Jonassen, M., Lund, H., San Agustin, H. and Sztuk, S. (2015). A gaze interactive textual smartwatch interface. Paper presented at the Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Com- puters, Osaka, Japan. https://doi.org/10.1145/2800835.2804332 [9] Harrison, C., Benko, H., & Wilson, A. D. (2011). OmniTouch: wearable multitouch interac- tion everywhere. Paper presented at the Proceedings of the 24th annual ACM symposium on User interface software and technology. https://doi.org/10.1145/2047196.2047255 [10] Harrison, C., Tan, D., & Morris, D. (2010). Skinput: appropriating the body as an input sur- face. Paper presented at the Proceedings of the SIGCHI conference on human factors in com- puting systems. https://doi.org/10.1145/1753326.1753394 [11] Jakobsen, M. R., & Hornbæk, K. (2011). Sizing up visualizations: effects of dis-play size in focus+context, overview+detail, and zooming interfaces. Paper pre-sented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Cana- da https://doi.org/10.1145/1978942.1979156 [12] Katsuragawa, K., Pietroszek, K., Wallace, J. R., & Lank, E. (2016). Watchpoint: Freehand Pointing with a Smartwatch in a Ubiquitous Display Environment. Pa-per presented at the Proceedings of the International Working Conference on Advanced Visual Interfaces, Bari, Italy. https://doi.org/10.1145/2909132.2909263 [13] Kerber, F., Kiefer, T., Löchtefeld, M., & Krüger, A. (2017). Investigating current techniques for opposite-hand smartwatch interaction. Paper presented at the Proceedings of the 19th In- ternational Conference on Human-Computer Inter-action with Mobile Devices and Services, Vienna, Austria. https://doi.org/10.1145/3098279.3098542 [14] Khakurel, J., Knutas, A., Melkas, H., Penzenstadler, B., Fu, B., & Porras, J. (2018). Categori- zation framework for usability issues of smartwatches and pe-dometers for the older adults. Paper presented at the International Conference on Universal Access in Human-Computer Interaction. https://doi.org/10.1007/978-3-319-92049-8_7 [15] Kurosawa, H., Sakamoto, D., & Ono, T. (2018). MyoTilt: a target selection method for smartwatches using the tilting operation and electromyography. Pa-per presented at the Pro- ceedings of the 20th International Conference on Hu-man-Computer Interaction with Mobile Devices and Services, Barcelona, Spain. https://doi.org/10.1145/3229434.3229457 [16] Likert, R. (1932). A technique for the measurement of attitudes. Archives of psychology. 142 http://www.i-jim.org https://doi.org/10.1145/2702613.2732837 https://doi.org/10.1016/j.chb.%202016.07.047 https://doi.org/10.1007/978-3-540-73287-7_57 https://doi.org/10.1145/2984511.2984563 https://doi.org/10.1145/2984511.2984563 https://doi.org/10.1145/2800835.2804332 https://doi.org/10.1145/2047196.2047255 https://doi.org/10.1145/1753326.1753394 https://doi.org/10.1145/1978942.1979156 https://doi.org/10.1145/2909132.2909263 https://doi.org/10.1145/3098279.3098542 https://doi.org/10.1007/978-3-319-92049-8_7 https://doi.org/10.1145/3229434.3229457 Paper—The Design and Evaluation of a New Smartwatch User Interface [17] McGuckin, S., Chowdhury, S., & Mackenzie, L. (2016). Tap 'n' shake: gesture-based smart- watch-smartphone communications system. Paper presented at the Proceedings of the 28th Australian Conference on Computer-Human Interac-tion, Launceston, Tasmania, Australia. https://doi.org/10.1145/3010915.3010983 [18] McMillan, D., Brown, B., Lampinen, A., McGregor, M., Hoggan, E., & Pizza, S. (2017). Situating Wearables: Smartwatch Use in Context. Paper presented at the Proceedings of the 2017 CHI Conference on Human Factors in Computing Sys-tems, Denver, Colorado, USA. https://doi.org/10.1145/3025453.3025993 [19] Narayanaswami, C., & Raghunath, M. T. (2000). Application design for a smart watch with a high-resolution display. Paper presented at the Digest of Papers. Fourth International Sympo- sium on Wearable Computers. https://doi.org/10.1109/iswc.2000.888452 [20] Nielsen, J. (1994). Usability engineering: Morgan Kaufmann. [21] Paul, C. L., & Bradel, L. (2018). Size matters: the effects of interactive display size on inter- action zone expectations. Paper presented at the Proceedings of the 2018 International Con- ference on Advanced Visual Interfaces, Castiglione della Pescaia, Grosseto, Italy. https://doi.org/10.1145/3206505.3206506 [22] Raptis, D., Tselios, N., Kjeldskov, J., & Skov, M. B. (2013). Does size matter? investigating the impact of mobile phone screen size on users' perceived usabil-ity, effectiveness and effi- ciency. Paper presented at the Proceedings of the 15th international conference on Human- computer interaction with mobile devices and services, Munich, Germany. https://doi.org/10.1145/2493190.2493204 [23] Rawassizadeh, R., Price, B. A., & Petre, M. (2014). Wearables: Has the age of smartwatches finally arrived? Communications of the ACM, 58(1), 45-47. https://doi.org/10.1145/2629633 [24] Smailagic, A., & Siewiorek, D. (2002). Application design for wearable and con-text-aware computers. IEEE Pervasive Computing, 1(4), 20-29. https://doi.org/10.1109/mprv.2002. 1158275 [25] Sridhar, S., Markussen, A., Oulasvirta, A., Theobalt, C., & Boring, S. (2017). WatchSense: On- and Above-Skin Input Sensing through a Wearable Depth Sensor. Paper presented at the Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, Colorado, USA. https://doi.org/10.1145/3025453.3026005 [26] Story, M.F. (1998). Maximizing Usability: The Principles of Universal Design, Assistive Technology: The Official Journal of RESNA, 10:1, 4-12. https://doi.org/10.1080/10400435. 1998.10131955 [27] Thorp, E. O. (1998). The invention of the first wearable computer. Paper present-ed at the Digest of Papers. Second international symposium on wearable com-puters (Cat. No. 98EX215). https://doi.org/10.1109/iswc.1998.729523 [28] Xia, H., Grossman, T., & Fitzmaurice, G. (2015). NanoStylus: Enhancing input on ultra-small displays with a finger-mounted stylus. Paper presented at the Pro-ceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. https://doi.org/10.1145 /2807442.2807500 [29] Yeo, H.-S. (2018). Enabling Single-Handed Interaction in Mobile and Wearable Computing. Paper presented at the The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings, Berlin, Germany. https://doi.org/10.1145/ 3266037.3266129 iJIM ‒ Vol. 15, No. 13, 2021 143 https://doi.org/10.1145/3010915.3010983 https://doi.org/10.1145/3025453.3025993 https://doi.org/10.1109/iswc.2000.888452 https://doi.org/10.1145/3206505.3206506 https://doi.org/10.1145/2493190.2493204 https://doi.org/10.1145/2629633 https://doi.org/10.1109/mprv.2002.1158275 https://doi.org/10.1109/mprv.2002.1158275 https://doi.org/10.1145/3025453.3026005 https://doi.org/10.1080/10400435.1998.10131955 https://doi.org/10.1080/10400435.1998.10131955 https://doi.org/10.1109/iswc.1998.729523 https://doi.org/10.1145%0b/2807442.2807500 https://doi.org/10.1145%0b/2807442.2807500 https://doi.org/10.1145/3266037.3266129 https://doi.org/10.1145/3266037.3266129 Paper—The Design and Evaluation of a New Smartwatch User Interface 7 Authors Dr. Pietro Murano is an Associate Professor of Computer Science, OsloMet – Os- lo Metropolitan University, Department of Computer Science, universal Design of ICT Research Group, PO BOX 4, ST. Olavs Plass, 0130 Oslo, Norway. www.pietromurano.org. Email: pietro.murano@oslomet.no Karan Singh Chaudhary is a Master of Universal Design of ICT. OsloMet – Oslo Metropolitan University, Department of Computer Science, Universal Design of ICT Research Group, PO BOX 4, ST. Olavs Plass, 0130 Oslo, Norway. Email: krn4singh@gmail.com Article submitted 2021-03-17. Resubmitted 2021-0417. Final acceptance 2021-04-23. Final version published as submitted by the authors. 144 http://www.i-jim.org http://www.pietromurano.org/ mailto:pietro.murano@oslomet.no mailto:krn4singh@gmail.com