PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE The Implications of Mobile Notifications for User Experience of a Social Network Service http://dx.doi.org/10.3991/ijim.v7i2.2373 Sami Vihavainen1, Kaisa Väänänen-Vainio-Mattila2 1 Aalto University, Espoo, Finland 2 Tampere University of Technology, Tampere, Finland Abstract—Smartphones enable an always-on connection to Social Network Services (SNS). A typical way of interacting with SNSs is to access them when the user has a suitable situation to check the status of her social networks or to write an update. One way to enhance the usage of SNSs is to have the service automatically push notifications about events to the smartphone user interface. However, there is no research on how users experience such SNS notifications. We present an explorative field study with 11 participants to assess how users experience mobile notifications compared to reading SNS content manually, initiated by the user. The participants first used Facebook for a month without notifications and then for a month with an application called Socially that sends frequent notifications about Facebook events to the user’s smartphone desktop. The participants who kept the notification feature on reported increased reading of Facebook. However, after a while, many were unwilling to receive the notifications, mainly because of lack of control. We report the qualitative findings on user experience, which reveal for example, that the use of mobile notifications decreases interest in Facebook. Notifications limit user control, and using Facebook feels more meaning- ful when accessed manually. Implications for design are discussed. Index Terms—Mobile notification, Social Network Service (SNS), Field study, User experience. I. INTRODUCTION Social Network Services (SNSs) are increasingly a part of people’s everyday lives. They are used to maintain contact with acquaintances, both close ones and more distant ones [1]. Nowadays, SNSs are often used with smartphones. There are mobile SNS applications, for example for Facebook and Twitter, which enable people to follow their friends’ doings and socialize with them while on the move. Features of automation have been developed for SNSs, meaning that some tasks that were previously performed by the users are allocated to the SNS. For example, location sensors of a smartphone perform automatic “check-ins” on Foursquare [2]. This kind of automation reduces the need for users’ “manual” or user-initiated interaction with the service while supporting access to a broader set of information. Recently, automation has been offered also in the form of mobile notifications. Notification systems have been defined as “interfaces specifically designed to support user access to additional digital information from sources secondary to current activities” [3]. Mobile notifications inform the user about happenings on an SNS by pushing information on the smartphone desktop. However, earlier research has shown that from the user’s perspective, notifications are a double-edged sword. Notifications are valuable to the user in conveying important or relevant information, but they also come at the cost of interrupting the user [4]. How users experience notifications for SNSs has not been studied. To fill this gap, we studied the user experi- ence (UX) implications for mobile notifications on SNSs in the field. By user experience, we mean users’ subjec- tive experiences, which arise from user-system interac- tion in the context of use. User experience covers both pragmatic (practical, goal-oriented) and hedonic (emo- tional, non-instrumental) aspects of system use [5]. A field study with end users using a system or a service in real contexts of use is a strong method to gain under- standing of the details of user experience [6]. In studying mobile notifications, we focus on how ser- vice automatically follows happenings on the user’s SNS, analyzes the acquired data, decides what information to deliver, and delivers that information by pushing the information to the user’s phone as a notification. Thus, the system works as an assistant to follow activities of other people, which is a principal motivation for using services like Facebook [7]. We consider this to be an important area of research because notifications, if properly implemented, can enhance a user’s social interaction by assisting the user in being better aware of the happenings in her social circle. However, it is impor- tant to the user experience that the notifications’ interrup- tions are not perceived to be irritating. To study the user experience of mobile notifications, our main research question was as follows: How do mobile notifications affect the user experience of social network services? We conducted an explorative field study with end users using Facebook with an existing, relatively widely-used mobile application called Socially [8]. Socially pushes News Feed updates from users’ Facebook contacts to users’ mobile phone desktop based on predefined time intervals; 30 minutes was the default setting. It also gives a light sound and vibrates the phone each time a new notification pops up. With Socially, the user is also able to manually read the Facebook News Feed by opening the News Feed screen and looking through the updates. Our approach was to study a group of current Facebook users, introduce them to Socially, and gather and analyze data on how they use Socially in connection with Facebook. Our aim was to explore the user experience of mobile notifications and provide insight to designers to consider iJIM – Volume 7, Issue 2, April 2013 9 http://dx.doi.org/10.3991/ijim.v7i2.2373 PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE when designing SNS functionality in mobile contexts. From a broader viewpoint, we explored the potential of automatic features in the SNS context, i.e. how tasks can be transferred from a human to a machine in mobile social applications. In the remainder of this paper, we will first present related research followed by the research methods. We will then present the study process and the results. After discussing the design implications for mobile notifica- tions and automation for social media, we will provide conclusions the contributions of the study. II. RELATED RESEARCH In this section we examine research related to notifica- tions in Human-Computer Interaction (HCI). We also briefly present the concepts of automation and perceived control and how these concepts relate to notifications. Previous research has found that if properly imple- mented, notifications can support people’s awareness of others’ social states, actions, and activities [9]. However, notifications generate an interruption, and it is not self evident that they are perceived positively by users. The benefits of interruptions must outweigh the detriments [3]. McCrickard and Chewar [3] argued that users’ possible dissatisfaction with notifications is due to de- signers’ failure to estimate user task prioritization. This results in notifications presented at inappropriate times and in an unsuitable presentation style. Earlier research has also found that intensity, which can be “scaled from not notifying at all to trying explicitly to grab the entire user attention” (p. 3), is affected by the presentation modality and the amount of information presented to the user [4]. Notifications can be used also in mobile devices such as smartphones. However, the use contexts of mobile devices are diverse, and the user is normally performing some other primary task. Therefore, the notifications have to compete with the user’s environment to get the user’s attention [10]. User attention is limited; it has even been proposed that in today’s information society, attention is the scarcest resource [11]. Because a notification gener- ates an interruption, a notification appearing at the wrong time and too intensely generates annoyance. It is unwise to get the user’s attention at all costs. Several approaches to notifications, such as information filtering, choosing modality, and attentive UI, have been proposed to get a user’s attention in a satisfactory way [10]. The mobile context also sets requirements for notifications because the interaction events with mobile phones are often short and fragmented [12], such as checking Facebook happen- ings while on a bus [13]. In addition users accept notifica- tions more quickly if they are timed to appear at the end of mobile interaction episodes rather than at random times [14]. Recent research has also examined easing the user’s burden to keep up with SNS events by aggregating events from several SNSs. For example, Cui et al. [15] devel- oped and studied a LinkedUI application that enables the user to follow several SNSs in one consistent user inter- face. They found that the user preferred the aggregation approach to accessing SNSs through a traditional mobile web browser. LinkedUI decreased difficulties such as in switching windows and delays loading web pages. Overall, in the mobile and web environment, there are several ways to inform the user about changes taking place on SNSs or other web services. These change indicators can reduce the user’s burden in looking for new content [16]. Automation means that some aspects of human activity are transferred to the system. This also means that control is partially transferred from human to machine. Different levels of automation can be built into HCI [17], from full machine control to full user control. Notifications imple- ment a form of automation, which means that the ma- chine (in this case the SNS and/or the mobile device) takes control of when the user should start interacting with the SNS. Psychology research [18] has investigated the notion of control. Control refers to “the extent to which an agent can intentionally produce desired outcomes and prevent undesired ones. When individuals believe they can do this, they are said to have personal control, perceived control, or a sense of control” (p. 554). In information systems, perceived control has been found to affect users’ motivation to use the system. Novak et al. [19] found control to be a major determinant of the flow experience with online environments, which again affects the depth of interaction with the service. They defined control as the “user’s perception of her ability to successfully navigate through the Web environment and her percep- tion of how the Web responds to her inputs” (p. 27). On the other hand, lack of perceived control may cause desperation and hopelessness [20]. In the HCI context, it is important to avoid negative emotions, and that user has the impression that she is in control of situations [21]. In addition, people feel happier about the outcomes that they have accomplished themselves compared to similar outcomes that have been accomplished by someone else or by chance [20]. Earlier HCI research has looked into notifications and automation usage in domains other than SNSs. In psy- chology, perceived control has been explored as a con- cept that may improve the user’s experience with the system. In our study, we investigated how mobile notifi- cations, as a form of automation, affect the user experi- ence, including the user’s sense of control. III. SYSTEM DESCRIPTION In our study we used Facebook and Socially to study the user experience of mobile notifications. With So- cially, the user is able to read Facebook (and Twitter, Linked In, and Foursquare) by starting Socially and opening Facebook News Feed manually (Figure 1) or by reading notifications that the system pushes from Face- book (Figure 2). The user is also able to write status updates. Socially has other features that were not the focus of this study, including notifications during incom- ing calls, caller location, and synchronizing Facebook profile photos and birthdays with the phone. In our study, we used version 2.10. Based on the Facebook site, there were about 50,000 active users (27.10.2011), and the application had a 3.9/5.0-star average rating based on 173 reviews on Facebook. Socially scans the user’s social network services (e.g., Facebook or Twitter) and pushes new update notifications as a pop-up to the phone’s desktop (Figure 2). 10 http://www.i-jim.org PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE Figure 1. Socially’s Facebook News Feed reading view. Figure 2. Socially pushes recent updates to a phone’s desktop at predefined time intervals. When the new notification pop-up appears on the desk- top, the phone gives a short audio alarm and the phone vibrates. The audio alarm is about 0.5 second (knock- knock-knock). Volume follows the phone’s general ringing volume settings. Vibration cannot be turned off. It is possible to set the updates to come in time intervals of every 30 minutes, one hour, four hours, once a day, or never. The default setting has the phone push Facebook updates to the phone’s desktop every 30 minutes. The updates are the most recent updates that would appear on the user’s Facebook News Feed. When the notification appears, the user can scroll through the updates one by one, using the phone’s arrow keys or push Read or Exit. The maximum number of new updates to scroll through is 15 per notification. Pushing Read opens a view where the user can read and scroll through the entire News Feed, as shown in Figure 1. Pushing Exit closes the notification. It is also possible to shut the feature off either by turning off Desktop alerts or setting the update time interval to Never. Pushing the social notifications to the phone’s desktop makes the phone work for the user by keeping an eye on what is happening on the user’s SNS. This potentially enhances user’s awareness of her social surroundings. She does not have to manually check the Facebook News Feed; it is delivered to her phone’s desktop. IV. FIELD STUDY We recruited 11 participants for this qualitative study from the greater Helsinki area in Finland. We conducted a study in the field where we first studied participants’ Facebook use strategies and then the use of mobile notifications through Socially in concert with Facebook. A. Users and Devices The users were recruited from the SizzleLab [22] living lab environment. There were nine males and two females. All were young adults from 19 to 27 years of age (mean age = 23.3). All had used Facebook in their everyday lives before the study. Of the total, 10 had Nokia N97 phones, which they had used as their primary phones for at least six months before the study. One participant had an HTC smartphone with an Android platform. All the participants also had unlimited data plans. The phones and data plans were provided to the users for free. B. Data Collection 1) Field Study Use of an existing application was studied in the eve- ryday life context to better reveal the social and techno- logical issues that unfold only in long-term real life use and might not come up in laboratory and prototype conditions. The field study had two phases. In the first phase (Phase A), people’s Facebook use activities and strategies were studied. This phase lasted from 26 to 41 days (an average of 31 days per participant), depending on the users’ personal availability for the study. In the second phase (Phase B), we intervened and introduced the users to Socially. We asked the users to use Socially with Facebook. We instructed them to use the application as they wanted but encouraged them to at least try it. This phase lasted from 31 to 46 days, depending on the par- ticipant’s availability (an average of 41 days per partici- pant). A pseudo-Facebook profile was established for the study, and all the participants added the pseudo profile as their Facebook contact for the period of the study. This was done to log how often participants updated their Facebook profiles during Phase A and Phase B. Partici- pant privacy was taken seriously, and access to the pseudo profile was granted only to researchers who were part of the project group. All the data related to partici- pants was anonymized for publication. The participants installed Socially in their phones themselves. Because we wanted them to use the applica- tion as realistically as possible, we did not want to force them to use any of its features. However, all participants were introduced to all the features. They were asked to send us an e-mail or SMS check when they had success- fully installed the application, and we told them to ask for help if they had any problems. We provided technical support if needed. We did not reveal to them that the focus of the study was notifications. As a reward, each participant was given a 50€ fee at the end of the study. 2) Open-ended Questionnaires During the field study, two open-ended questionnaires were conducted. The first took place in the middle of Phase A and concentrated on the participants’ Facebook use habits and experiences. The second was conducted at the beginning of Phase B and concentrated on first iJIM – Volume 7, Issue 2, April 2013 11 PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE reactions and experiences with Socially and if the partici- pants had had any problems with it. Both questionnaires included real-life examples. At this point, we kept the questions quite general so that our research focus would not be revealed and so that the questions would not direct participants’ use of Socially. 3) Final Interviews and Questionnaires At the end of the study, we interviewed the partici- pants. Six participants were individually interviewed face-to-face. The five remaining participants were not available for interview. However, they were sent open- ended questionnaires, which were formulated based on the already conducted individual interviews. Interviews were semi-structured and consisted of reactions and use of the automated features of Socially. The interview protocol and the questionnaires covered the following areas: 1. Participant’s background (e.g., age, technology know-how) 2. General use of the phone (e.g., whether it worked properly) 3. General use of Socially (e.g., perceived usefulness) 4. Experiences with Socially’s notifications 5. Experiences reading Facebook News Feed manually 6. Whether they planned to use Socially after the trial and why C. Data Analysis Qualitative data from the open-ended questionnaires and face-to-face interviews was analyzed based on grounded theory as defined by Straus and Corbin [23]. This means that the theories or result themes are found bottom-up from the data, and no pre-study hypotheses are formed. Furthermore, qualitative data was analyzed using content analysis [24]. First, the key points related to data were identified. Then the key points were categorized into larger themes. Research questions were used to guide the analysis. Quantitative data on Facebook updates during Phases A and B was analyzed using a paired sample t-test. V. RESULTS In this section, we first describe users’ interaction with Facebook during Phases A and B (Section A) to provide background on the users’ Facebook habits and motiva- tions (Phase A) and general use of the notification feature (Phase B). Section B summarizes the user experiences with Socially. We then present the main result themes that arose from the study data in sections D–G. A. Facebook Interaction in Study Phases A and B 1) Phase A—Reading News Feed and Hanging Out For most of the participants, reading was the main ac- tivity on Facebook; motivations varied from a need to keep up with happenings to a need to relax. Many claimed that they use Facebook mainly to follow what their friends are doing or for private chatting and mailing, but they are passive in writing about their doings and whereabouts in the public status box. According to our questionnaire, on average, they read Facebook about 5.1 times per day. However, some were clearly more active than others; the standard deviation was 4.6. As everyone reported that they read Facebook every day, the users were active readers of Facebook. Regarding Facebook contribution, we counted the times they had written a status update or sent a link (e.g., YouTube video or a web site) to their profile during Phase A. The total number of updates was 86. On aver- age, the participants sent 0.26 updates per day; the stan- dard deviation was 0.38. One of the users (P8) was a very active contributor, sending 1.2 updates per day on aver- age, whereas five participants sent no updates. Using Facebook seemed more serious to some than to others. For example, one participant described how he needs Facebook to be up to date on his friends’ doings and new events. “[Without Facebook] I would be so clueless about everything what happens.” – P2 But many described their use as “hanging out”. It was popular among the participants to read Facebook when there was nothing else to do. It was more about relaxing and passing the time than serious objectives. Using Facebook while on the move was popular. Half of the respondents reported that they have a habit to read Face- book while on a bus or a train. Nine participants had used Facebook with their phones before the trial. The majority reported reading as the main activity with the phone. The participants’ basic Facebook habits were in line with earlier studies of SNS uses [7] and interaction moments with the service [13]. 2) Phase B—Differences in the Use of Notifications Overall, participants differed on whether they used the notification feature offered by Socially. Three partici- pants (P1, P2, P6) kept the feature on for the entire use period (about 4 weeks) while six (P3, P4, P5, P8, P9, P10) kept it on from one to seven days; two (P7, P11) did not use it at all. Next, we will discuss these differences in the use of notifications. We will also elaborate on the findings by comparing them to earlier research. We will refer to the above-mentioned groups as High-users, Low- users, and Non-users. P1, P2, and P6 belong to the High- users; P3, P4, P5, P8, P9, and P10 belong to the Low- users; and P7 and P11 form the Non-users group. Non- users had problems understanding the logic of how the notifications feature works. P7 accidentally deactivated the notifications after installation. She deactivated the “Desktop Alerts” feature without realizing that it also prevented notifications from showing up. P11 had the feature on but he did not keep Socially running in the background. He did not realize he needed to keep the application running to be able to use automated features. B. The General Effect of and User Experience with Socially 1) The Main Effect of Socially: Reading Increased But Writing Did Not Six participants (P2, P3, P4, P5, P6, and P10) reported that Socially increased their Facebook reading but did not have a clear effect on how they updated their status lines. Four (P1, P7, P8, and P11) reported that Socially did not have a significant effect on how or how much they used Facebook; one (P9) reported it actually reduced her Facebook activity while the notifications were on. The most important reasons participants reported for increasing reading activity was that it was quicker to 12 http://www.i-jim.org PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE manually read the Facebook News Feed with Socially than how they had previous read the Feed; and because of the notifications (when turned on), they were persuaded to read the updates more often. In total, 160 updates were sent during Phase B, and only 7 were written using Socially. On average, participants sent 0.35 updates per day, with a variance of 0.50. On average, there were more updates sent per day in phase A (0.26), but the difference was not significant. Based on paired samples t-test, the difference between Phases A and B was not significant; the p-value was 0.078. 2) User Preferences and Experiences with Socially Most of the participants reported that they liked So- cially. Six participants (P2, P5, P6, P7, P10, and P11) planned to continue using Socially after the trial period. Three (P4, P8, and P9) were a bit uncertain, and two (P1 and P3) would not use it after the trial. The main reasons for continuing the use were notifications and ease of use. For three participants, notifications were one of the main reasons they wanted to continue using Socially. However, one (P11) did not understand that Socially had to be kept in the background. After he realized that, he wanted to continue using the system so that he could receive notifi- cations. For three, the main reason was that Socially was easier and faster to use than other ways (web browser or Nokia’s Facebook application). With Socially, it was convenient to open the application manually and pull the Facebook updates to the phone. Most found that from a pragmatic perspective, Socially worked well. It was fast and easy to use. However, from a hedonic perspective, there was more variation among the participants. Many found that notifications were not pleasant and did not suit their Facebook use habits whereas some liked felt they were a bonus. C. The Main Themes Related to User Experience of Mobile Notifications During the analysis phase, four main themes related to user interaction with mobile notifications came up: Users had varying control strategies over notification interrup- tions (1); ease of skipping insignificant messages favored manual reading (2); notifications decreased interest in reading Facebook (3); and users’ goals affected the perceived usefulness of notifications (4). 1) Users Have Varying Control Strategies over Interruptions The effect of practicing control over notification fre- quency and intensity came up when comparing the High- users and the Low-users. Two from the High-users had made the update interval less frequent than the default setting of “every 30 minutes” (to “every four hours”). For them, notifications were actually one of the main reasons they wanted to continue using Socially after the study. One was bothered by the intensity (alarm) of the notifica- tions. He wanted to keep the notification feature on, but because he could not turn off the alarm, he chose to lower the sound level of his phone. The third user who kept the notification feature on used the 30-minute default interval for the entire period. However, he kept his phone silent all the time. He said that he always keeps his phone on silent because the phone’s alarms generally disturb him. For him, the notifications were not as intense as for others. But he did keep the phone’s vibration feature on to be aware of incoming messages and phone calls. Of the six Low-users, none had practiced any control over the mechanisms of the notifications. All of them used the 30- minute default until they turned the feature off. There- fore, although the system had a mechanism for the user to adjust the frequency of notifications and to control interruptions based on weighting the value of interruption against the costs, they did not use this mechanism. They also did not control the intensity of notifications by adjusting the phone’s sound level like two of the High- users did. The Low-users thus perceived the notifications as more intrusive than the High-users. In addition to controlling the settings of the notifica- tions, varying levels of behavior control practiced by the participants were reported. Two of the High-users said they read the updates practically every time they showed up unless they were in a hurry. But even they did not actually read the new status lines every time even though they might have checked the pop-up. ”But it’s a different thing what I will actually do after I have opened the key lock. Will I push Read or Exit? It depends a lot on what else am I doing at the time.” – P2 They managed to control the notification situations with their behavior: They simply did not pay much attention to notifications when they did not want to read them. Even unimportant notifications did not cause interruptions so severe that it would have irritated them as they were in the middle of their primary tasks. Some of the Low-users related how they at first thought that the feature was pretty cool, but that after a short while they realized they would not need or want notifications about new Facebook updates but wanted to control their Facebook reading manually. “I tried the feature for about three days but then it started to get on my nerves, when it beeped every day. I thought this is not for me because I like to go to Face- book often and watch the new updates by myself any- way.” – P5 “Those kinds of things [notification features] are really neat. I was really excited at first, but then again in the long run it was not that good anyway.” – P10 This brings up how the user experience can change over time. After the notifications were first considered useful, withdrawal occurred. Unlike the High-users, most of the Low-users did not report practicing much behavior control. Of the Low-users, five reported that they checked the notifications very often before inactivating them. Only one never really checked the updates but just ignored the update alerts. Low-users said that the alarm sounds Socially gave when sending notifications to the phone’s desktop were disturbing. The alarm was too persuasive and one participant mentioned how she re- acted physically to the alarms. “I reacted perhaps quite physically to it. Like, hey! Now it alarms, I have to check it out.” – P5 To ensure that the push alerts do not bother the user, Socially automatically stops showing updates at night. This window is from 9PM to 8AM. Although this was a nice feature, it was not enough because it did not adapt to the user’s real life well enough. In the following case, the user’s strategy to control notifications was eventually to shut off the notification feature. iJIM – Volume 7, Issue 2, April 2013 13 PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE “I really do not want to wake up during the weekend mornings because some half-acquainted person has added a YouTube-video to her Facebook wall. – P5 Earlier research [4] has shown that the intensity of notifi- cations is influenced by the modality in which the notifi- cations are presented, including whether they are silent or not. Based on the results, frequency seems to be a defin- ing factor of the perceived intrusiveness of the notifica- tion system. The control possibilities for the user to adjust the intensity of notifications should not be overlooked or made too difficult. Users do not necessarily explore the control possibilities spontaneously but may turn the system completely off if controlling the notification is not easy enough. 2) Ease of Skipping Insignificant Messages Favors Manual Reading Many participants reported that they wanted to control their Facebook reading manually by scrolling through the News Feed. This was because they felt that it gave them better control over what to focus on. There were two main reasons notifications were not perceived as a valu- able way to follow Facebook. The social importance of one’s Facebook contacts varied widely. However, notifi- cations treated all the updates as equally significant. For example, one user had about 1000 Facebook friends and was not interested in getting all of those friends’ status updates to his phone; for instance, he only gets one-to- one text messages that are directed only to him. Some participants had blocked some of their Facebook friends’ status updates because they perceived that many were useless. The fact that this could not be done with Socially was disturbing. Several participants stated that the greater the user control in defining the filtering principles of Facebook status lines, the more they would like the notification feature. Second, mobile notifications empha- sized individual messages whereas the News Feed shows several updates at the same time. With the entire News Feed on display, it is easier to control the unwanted, boring, and useless status updates by just skipping up- dates. It was considered important to control pre-filtering of updates and filtering during the reading process. Participants thought that there was a lot of unnecessary “noise” on Facebook that was not considered meaningful. Many were irritated that the notifications did not provide a meaningful extent to which user was able to intention- ally produce desired outcomes and prevent undesired ones. Showing one update at a time makes the first update especially important. Reading the first update can be a crucial moment of interaction when the user makes the decision to continue reading Facebook. One of the par- ticipants mentioned that when he gets notifications, he checks the first status update and based on that decides whether to explore the updates further or stop the prevail- ing interaction event. “If that first one [status update] is something that does not interest me at all, I might push Exit and check the others sometime later just by doing it manually.” – P2 Another user had actually never read other than the first one. “I never continued reading the updates after the first one, because already the first one seemed useless.” – P5 We consider this to be a valuable insight for designers. If the first shown status update were the most commented on or most liked, it might trigger a deeper interaction event and persuade the user to continue reading Face- book. 3) Notifications Decrease Interest in Facebook Losing control of what to read and when decreased user interest in Facebook. For some users, notifications made the content of Facebook updates feel less meaning- ful. One of the Low-users noted that during the first days of use, she checked the updates every time. However, when reading her friends’ status updates from the phone, she realized how irrelevant they were to her current context. After that “enlightenment”, she said she used Facebook less than she would have normally because she learned to ignore the notification alarms but did not read Facebook manually from phone or from a desktop com- puter. She thought that there was nothing interesting on Facebook to read, but when she turned the feature off (after keeping it on for a week), she started to read Facebook from her desktop computer again. “During that one week I opened Facebook from my computer only like two times. That was because all the updates came to the phone. But somehow I then noticed that if these all are this kind of crap, why do I bother reading them at all. However, after I stopped the push feature I started to use Facebook more often with a computer.” – P9 For her, the push alarms were eventually annoying and made her uninterested in Facebook. “It is weird, that suddenly it is really interesting if you do it by yourself, but not if someone pushes it to you.” – P9 Another participant described similar feelings when reading Facebook manually and getting notifications. For me it feels somehow more meaningful to check the updates manually, than if I am just being told [through notifications]. – P1 Psychology research has produced results similar to these findings. As previously mentioned, people feel happier about outcomes that they have accomplished themselves compared to similar (equally pleasant) out- comes that have been accomplished by someone else or by chance [20]. This might partly explain why reading the News Feed manually was somehow more meaningful than getting the same content delivered through notifica- tions. During the manual reading process, the user ac- complishes the interaction event whereas through notifi- cations, something else is the primary agent accomplish- ing the interaction event. 4) User Goals Affect Perceived Usefulness of Notifications Participants’ Facebook use motivations and the goals they try to achieve by using Facebook affected how useful the notification feature was perceived. The High- users who perceived that the notifications enhanced the user experience of the system were keen on being aware of what their Facebook friends were up to. Two of the High-users discussed how Facebook was all about know- ing what friends are doing and what kind of social events have occurred, are occurring, or will be occurring. One was actually surprised when we asked if the notifications had bothered him. The other felt that the notifications 14 http://www.i-jim.org PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE were one of the main reasons he will continue using Socially after the trial. The third User (who kept his phone silent) explained that for certain intervals, notifica- tions are a nice bonus. Overall, these users perceived that notifications helped them be more aware of the happen- ings in their social network. The value was greater than the detriment of notification interrupting the current primary task. Using the user experience related notions presented by [5]; for the High-users notifications en- hanced the hedonic quality of Socially. This was because they wanted to be aware of others’ social lives, and for this notifications were useful. Notifications enhanced the user experience of the service by explicitly supporting users’ hedonic goal of being related to others. These users were more focused on following others than the partici- pants who did not perceive notifications as useful and who used Facebook to pass the time and relax. Four of the Low-users found that they did not want to enhance their Facebook use in general. Ironically, this is exactly what the notifications try to do. Although all the Low-users used Facebook quite actively in their everyday lives (as discussed in 4.3), it was not a high priority and was used to pass the time or relax. This was a significant reason why many of the Low-users did not like notifica- tions. Therefore, the implementation of notifications was not in line with the users’ hedonic goals of passing the time and relaxing. The Low-users perceived that their control of when to read Facebook was too limited and did not feel that automation supported their use goals. One of the Low-users thought that notifications take too much attention from more important tasks like studying. “If one should be studying, and Facebook is such a temptation anyway and easy to get stuck with, these [notifications] just increase that danger." – P10 VI. DISCUSSION AND DESIGN IMPLICATIONS It can be tempting for designers to add automated fea- tures to social media applications. Potentially, automation can increase the amount of social interaction and enhance the user experience. However, the findings of this study show that automation in the social media context is not all about efficiency and productivity. Concerning notifi- cations, the user has to feel that notifications support motivations and goals in using the service and that the benefits overweight the intrusiveness of the generated interruptions. It is also important that the user has control over the intrusiveness of notifications. Careful considera- tion is required when implementing features that push SNS content to people’s personal phones. Next, we will discuss our findings and offer recommendations on how to design automation in a mobile notification context. We will also discuss the broader implications of automation for mobile social applications. A. Use Goals Are Significant in Defining Social Media UX People can have various motivations for reading about happenings on SNSs like Facebook. Some might feel it very important not to miss information about their friends’ doings whereas others might read SNSs only when they do not have anything else to do or want to take a relaxation break from more important tasks. Notifica- tions made it efficient to be up to date on Facebook happenings. It did assist in the pragmatic goal of getting information. However, the use goals played a significant role in user experience. Designers should thus concen- trate on supporting users’ hedonic goals and not only pragmatic goals. As a concrete example, the service could have a feature that when first used would ask the user questions concerning her goals related to Facebook use and set up notification settings based on the user’s an- swers. B. Losing Control May Reduce Motivation to Follow SNS Updates As the functions are automated, control is transferred from human to machine. This may lower the user’s perceived control of following SNSs. Several participants reported that reading the updates manually made the information feel more meaningful, and they felt more motivated to read Facebook manually than by using mobile notifications. With notifications, the trigger for checking SNS updates comes from a machine. For most users, it was better for the trigger to come from the user herself. Notifications can actually decrease reading activity compared to manual reading, as was reported by one of the participants. In information systems, perceived control has been found to affect users’ motivation to use the system [19], and the lack of autonomy of actions has been shown to reduce motivation [25]. Research indicates that people prefer accomplishing outcomes themselves to having them accomplished by someone else [20]. So it is important for designers of mobile notifications to con- sider the importance of perceived control for user experi- ence. C. Notifications Can Violate the Privacy of Information Input Users can interpret notifications as an invasion of pri- vacy. This occurs when privacy is understood not only as a possibility for the individual to control what other people know about her (output of information) but also the individual’s ability to control the information flow about others to him or her (input of information). This input-output distinction of privacy has been presented by [26]. With notifications, user control of information input might be in danger, and lack of control can be perceived as a violation of privacy. For example, user privacy was violated when she was not able to sufficiently control getting a notification at 9 A.M. on Sunday morning about some half-acquaintance linking a YouTube video to Facebook. Her strategy of controlling her privacy was to shut off the notifications feature completely. We propose that when developing systems that suggest something that is not a high priority for the user, designers should con- centrate on situations in which the user already interacts with his or her phone. For example, notifications could show up only when the user actively uses the phone. Thus, a severe interruption could be avoided. However, in these situations, it is important that the notification does not stand out too forcefully so it does not complicate the primary task the user is performing. D. Activity recognition can support relevance of notifications Activity recognition has been proposed as a way to automatically find proper moments for interruptions (e.g., [27] and [4]). The fast development of smartphones and ubiquitous computing has enabled smartphones to contain numerous sensors that acquire data on a user’s physical surroundings as well as interaction with the phone and iJIM – Volume 7, Issue 2, April 2013 15 PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE phone’s interaction with surrounding devices. For exam- ple, ContextPhone platform developed in 2005 records data from 16 different types of sensors that could then be used to trigger actions within a service that uses the data [28]. A more recent example is the Android operating system-based Funf [29], which is able to record data from over 30 different built-in mobile phone data probes [30]. It might be useful for developers of notification features to try to use built-in sensors such as an accelerometer, magnetometer, and GPS sensor to model the activities of the user. Such a sensor-based approach could reveal users’ activities, such as whether a user is on a bus or cycling. Successful recognition of activities (see, e.g., [31], [32]) might enhance suitable moments for notifica- tions. However, although this kind of adaptive automa- tion might create a better user experience, it is likely that the user control is needed. We still (after tremendous advances in machine learning) seem to be in the situation where humans are vital to systems with automation [33]. D. Repetition Frequency of an Automated Task Should Not Be Extreme by Default Designing automation in the mobile social context is not only about allocating tasks between humans and machines but also about how to inform the user about the possibilities and workings of automation. For example, if the settings of automated features could be changed by the user but the default setting is at one extreme (i.e., very frequent or very rarely performed automated features), the user might not even be aware of the possibility of delegating tasks to the phone, or the user might become so irritated because of intense automation that he or she turns the feature off. This happened with many of the participants — they got irritated partly because the pop- up frequency of notifications was so high. Notifications might have been perceived more positively if the default pop-up frequency had been, for instance, every two hours. E. The Time Window for Automation to Assure the User of Its Usefulness Is Short It took at most one week for users to decide whether the notifications feature was useful. The majority of the participants were positive about the feature beforehand, and they thought that getting the updates automatically to their phones’ desktops was nice. However, after a week, the majority had turned the feature off because it did not adequately accommodate their needs and Facebook use. We believe this window is extremely critical in auto- mated systems that in some way cause interruptions in mobile contexts. Although more data is needed to define a precise time frame during which users either approve or disapprove of the feature, the implication for design is that designer should make the system adapt to a user’s usage habits fairly quickly. For example, after noticing that a user does not browse and read the pushed Facebook updates, Socially could automatically suggest alternative notification settings (e.g., lower notification frequency or silent notifications). F. Users Must Be Able to Choose the Level of Automation Based on the Significance of Information In SNSs, the sizes of social networks and the closeness of the contacts vary. For example, some of the partici- pants had over 1000 Facebook contacts whereas some had about 100. Raising 1000 contacts’ Facebook updates almost to the level of personal text messages was not perceived as useful. Only important messages directed specifically to the user were designated to be pushed to the phone automatically. The current iPhone and Android Facebook applications let users manage notifications by selecting what kind of messages the user wants as notifi- cations, such as friend requests, events, or likes. How- ever, the other solution could be to use varying levels of automation depending on the significance of information. Automation does not have to be all or nothing. For example, highly relevant information could include an alarm whereas information of less relevance would not, and the least relevant information would not be pushed as a notification at all. Another important consideration is of course how to determine what is significant and what is not. One possi- bility would be that the user selects the automation level for different kinds of messages (e.g., friend requests, likes, etc.). Another possible direction for information filtering could be to combine automatic filtering (e.g., a crowdsourcing approach in which the system automati- cally filters out updates that have not been read or are not considered important by one’s peers) and user control and lowering the level of automation by giving the user the option to include his or her own criteria (e.g., in form of tags) in the filtering algorithm. Regarding the parts of the selection process where the user is not in control, a discussion about the values of design would be necessary. For example, the developer might want to raise the importance level of updates that contain names of some brands, meaning that notifications could be considered mobile ads. Making these kinds of decisions without informing the user might be ethically dubious and might also lower the user’s perceived control of what to read on SNSs. VII. LIMITATIONS OF THE STUDY There are three limitations of this study. First, the number of participants was small. This qualitative study emphasized finding implications for designs using mobile notifications. Our aim was primarily to understand the details of the user experience, so we chose the qualitative approach. With a larger sample size, additional results on the quantitative effect of notifications on usage amounts could have been established. It is also possible that a larger user sample would have revealed further user innovations on how to use notifications to limit excessive Facebook use, such as by reading Facebook only when a notification pops up. Second, the results of the use and the effect of notifications were self-reported. Before the study, we tested and considered a context logging appli- cation to run in the background. However, because the Socially application was not our own but a commercial one, we did not have access to the source code, and existing context logging systems were not able to log quantitative data on the use of notifications. These limita- tions open up an area for future research. Logging the use of notifications of a large group of users could make it possible to quantify their effect in more detail. Third, as Socially’s notifications were fairly intense (frequent), they can be considered as a somewhat extreme case of mobile notifications. Thus, many participants stopped using notifications after some days. However, the inten- sity of notifications as the default enabled us to find 16 http://www.i-jim.org PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE results related to user control and experience, which might not have shown with less intense notifications. VIII. CONCLUSIONS It is apparent that having enough perceived control over automation-assisted reading of an SNS is essential for a good user experience. Most people who used the mobile notifications reported increased reading of Face- book. However, after using Socially notifications for a while, many were unwilling to receive them and felt the system limited their control too much. Whereas some participants perceived that notifications helped them be better aware of their social surroundings, others noted a decreased interest in Facebook. Having a manual means to control Facebook reading felt more suitable for varied everyday use. We believe that it is valuable for designers to understand that people will practice control over the notification system one way or the other to suit their SNS use purposes. Offering easy-to-use technical ways for the user to adjust the intensity and content of notifications assists the adaptation of the notification system to indi- vidual needs. Inaccurate, impractical, or limited technical ways to personify the notification system might make users control them by shutting them down or ignoring notifications completely. In this paper, we suggested design implications for better user experience of mobile notifications. A valuable direction for future work would be to study the best and most convenient ways to ensure that the intensity and value of notifications meets the use practices of an individual SNS user. Considering the rapidly-developing sensing capabilities of modern smart- phones, we see activity recognition as an interesting path to study the possibilities of automatically finding the proper contexts for notifications. However, at the same time, the importance of perceived control to a good user experience must be kept in mind in the design process of SNSs. ACKNOWLEDGMENT We would like to thank Erkki Kuisma and Hannu Vilpponen from Nokia Research Center for supporting the study, Antti Oulasvirta for support in the early phase of the study design, and Risto Sarvas for comments on the early version of the manuscipt. REFERENCES [1] E. Gilbert and K. Karahalios, “Predicting tie strength with social media,” in Proceedings of the 27th international conference on Human factors in computing systems, 2009, pp. 211–220. [2] Auto Check In Lite. (2012). foursquare. Retrieved November 9, 2012, from https://play.google.com/store/apps/details?id=com.rw mthings.android.autocilite [3] D. S. McCrickard and C. M. Chewar, “Attuning notification design to user goals and attention costs,” Communications of the ACM, vol. 46, no. 3, pp. 67–72, 2003. http://dx.doi.org/10.1145/ 636772.636800 [4] N. Kern and B. Schiele, “Context-aware notification for wearable computing,” in Proceedings of the 7th IEEE International Sympo- sium on Wearable Computers (ISWC’03), 2003, pp. 223–230. [5] M. Hassenzahl, The thing and I: Understanding relationship between user and product. In: Blythe, M.A., Overbeeke, K., Monk, A.F., and Wright, P.C. (Eds.) Funology: From usability to enjoyment. Springer, 2003. pp. 31-42. [6] D. J. Lazar, D. J. H. Feng, and D. H. Hochheiser, Research methods in human-computer interaction. West Sussex, UK: John Wiley & Sons, 2010. [7] A. N. Joinson, “Looking at, looking up or keeping up with people?: motives and use of Facebook,” in Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, New York, NY, USA, 2008, pp. 1027–1036. http://dx.doi.org/10.1145/1357054.1357213 [8] Socially mobile client. (2012). Retrieved November 7, 2012, from http://www.sociallyapp.com/ [9] J. M. Carroll, D. C. Neale, P. L. Isenhour, M. B. Rosson, and D. S. McCrickard, “Notification and awareness: synchronizing task- oriented collaborative activity,” International Journal of Human- Computer Studies, vol. 58, no. 5, pp. 605–632, May 2003. http://dx.doi.org/10.1016/S1071-5819(03)00024-7 [10] P. Isomursu, M. Isomursu, and M. Ervasti, “User Attention in Mobile Devices,” presented at the ACHI 2012, The Fifth Interna- tional Conference on Advances in Computer-Human Interactions, 2012, pp. 161–164. [11] T. H. Davenport and J. C. Beck, The attention economy: Under- standing the new currency of business. Harvard Business Review Press, 2002. [12] A. Oulasvirta, S. Tamminen, V. Roto, and J. Kuorelahti, Interac- tion in 4-second bursts: The fragmented nature of attentional re- sources in mobile HCl. In Proceedings of CHI 2005 pp. 919-927. 2005. [13] A. Oulasvirta, T. Rattenbury, L. Ma, and E. Raita, “Habits make smartphone use more pervasive,” Personal and Ubiquitous Com- puting, vol. 16, no. 1, pp. 105–114, 2012. http://dx.doi.org/ 10.1007/s00779-011-0412-2 [14] J. E. Fischer, C. Greenhalgh, and S. Benford, “Investigating episodes of mobile phone activity as indicators of opportune mo- ments to deliver notifications,” in Proceedings of the 13th Interna- tional Conference on Human Computer Interaction with Mobile Devices and Services, 2011, pp. 181–190. [15] Y. Cui, M. Honkala, K. Pihkala, K. Kinnunen, and G. Grassel, “Linked Internet UI: A mobile user interface optimized for social networking,” in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services, 2010, pp. 45–54. http://dx.doi.org/10.1145/1851600.1851611 [16] K. Väänänen-Vainio-Mattila and M. Wäljas, “How do users find out what’s new: a study of change indicators in mobile services,” in Proceedings of the 15th International Academic MindTrek Con- ference: Envisioning Future Media Environments, 2011, pp. 201– 204. [17] R. Parasuraman, T. B. Sheridan, and C. D. Wickens, “A model for types and levels of human interaction with automation,” IEEE Transactions on Systems, Man and Cybernetics, Part A, vol. 30, no. 3, pp. 286–297, 2000. http://dx.doi.org/10.1109/3468.844354 [18] E. A. Skinner, “A guide to constructs of control.,” Journal of personality and social psychology, vol. 71, no. 3, p. 549, 1996. http://dx.doi.org/10.1037/0022-3514.71.3.549 [19] T. P. Novak, D. L. Hoffman, and Y. F. Yung, “Measuring the customer experience in online environments: A structural model- ing approach,” Marketing Science, vol. 19, no. 1, pp. 22–42, 2000. http://dx.doi.org/10.1287/mksc.19.1.22.15184 [20] P. C. Ellsworth, Some reasons to expect universal antecedents of emotion. In P. Ekman & R. J. Davidson (Eds.), The nature of emo- tion: Fundamental questions. New York: Oxford University Press, 1994, pp. 150–154. [21] S. Brave and C. Nass, “Emotion in human-computer interaction,” The human-computer interaction handbook: fundamentals, evolv- ing technologies and emerging applications. New York: Taylor & Francis, pp. 81–96, 2003. [22] SizzleLab. (2012). Retrieved November 9, 2012, from http://mide.aalto.fi/en/OtaSizzle [23] A. C. Strauss and J. M. Corbin, Basics of qualitative research: grounded theory procedures and techniques, 2nd ed. Sage Publi- cations, Inc, 1990. [24] Y. Zhang and B. M. Wildemuth, “Qualitative analysis of content,” Applications of social research methods to questions in informa- tion and library science, Westport, CT: Libraries Unlimited, pp. 308–319, 2009. [25] R. M. Ryan and E. L. Deci, “Intrinsic and extrinsic motivations: Classic definitions and new directions,” Contemporary educa- iJIM – Volume 7, Issue 2, April 2013 17 https://play.google.com/store/apps/details?id=com.rwmthings.android.autocilite https://play.google.com/store/apps/details?id=com.rwmthings.android.autocilite http://dx.doi.org/10.1145/636772.636800 http://dx.doi.org/10.1145/636772.636800 http://dx.doi.org/10.1145/1357054.1357213 http://www.sociallyapp.com/ http://dx.doi.org/10.1016/S1071-5819(03)00024-7 http://dx.doi.org/10.1007/s00779-011-0412-2 http://dx.doi.org/10.1007/s00779-011-0412-2 http://dx.doi.org/10.1145/1851600.1851611 http://dx.doi.org/10.1109/3468.844354 http://dx.doi.org/10.1037/0022-3514.71.3.549 http://dx.doi.org/10.1287/mksc.19.1.22.15184 http://mide.aalto.fi/en/OtaSizzle PAPER THE IMPLICATIONS OF MOBILE NOTIFICATIONS FOR USER EXPERIENCE OF A SOCIAL NETWORK SERVICE tional psychology, vol. 25, no. 1, pp. 54–67, 2000. http://dx.doi.org/10.1006/ceps.1999.1020 [26] I. Altman, The environment and social behavior: Privacy, personal space, territory, and crowding. Monterey: Brooks/Cole Publishing Company, 1975. [27] J. Fogarty, S. E. et al. Predicting human interruptibility with sensors. ACM Transactions on Computer-Human Interaction, 12, 2005, pp. 119-146. http://dx.doi.org/10.1145/1057237.1057243 [28] M. Raento, A. Oulasvirta, R. Petit, and H. Toivonen, “Con- textPhone: A prototyping platform for context-aware mobile ap- plications,” IEEE Pervasive Computing, pp. 51–59, 2005. http://dx.doi.org/10.1109/MPRV.2005.29 [29] N. Aharony, W. Pan, C. Ip, I. Khayal, and A. Pentland, “Social fMRI: Investigating and shaping social mechanisms in the real world,” Pervasive and Mobile Computing, vol. 7, no. 6, pp. 643– 659, Dec. 2011. http://dx.doi.org/10.1016/j.pmcj.2011.09.004 [30] Funf. 2012. http://funf.org/journal.html. Retrieved 9 November 2012. [31] J. R. Kwapisz, G. M. Weiss, and S. A. Moore, “Activity recogni- tion using cell phone accelerometers,” SIGKDD Explor. Newsl., vol. 12, no. 2, pp. 74–82, Mar. 2011. http://dx.doi.org/10.1145/ 1964897.1964918 [32] S. Reddy, M. Mun, J. Burke, D. Estrin, M. Hansen, and M. Srivastava, “Using mobile phones to determine transportation modes,” ACM Transactions on Sensor Networks, vol. 6, no. 2, pp. 1–27, Feb. 2010. http://dx.doi.org/10.1145/1689239.1689243 [33] R. Parasuraman and C. D. Wickens, “Humans: Still vital after all these years of automation,” Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 50, no. 3, p. 511, 2008. http://dx.doi.org/10.1518/001872008X312198 AUTHORS Sami Vihavainen is with the Aalto University’s Hel- sinki Institute for Information Technology HIIT, P.O. Box 15600, 00076 Aalto, Finland (e-mail: sami.vihavainen@hiit.fi). Kaisa Väänänen-Vainio-Mattila is with the Tampere University of Technolgy’s unit of Human-Centered Technolgy, department of Pervasive Computing, P.O. Box 589, 33101 Tampere, Finland (e-mail: kaisa.vaananen-vainio-mattila@tut.fi). Sami Vihavainen was supported by the Academy of Finland project AMOVEO and Aalto University’s project Otasizzle. Otasizzle project provided the living lab environment. Nokia Research Center also supported the study by providing a smart phone for the researachers to use during the study. Received 22 November 2012. Published as resubmitted by the authors 20 March 2013. 18 http://www.i-jim.org http://dx.doi.org/10.1145/1057237.1057243 http://dx.doi.org/10.1109/MPRV.2005.29 http://dx.doi.org/10.1016/j.pmcj.2011.09.004 http://funf.org/journal.html http://dx.doi.org/10.1145/1964897.1964918 http://dx.doi.org/10.1145/1964897.1964918 http://dx.doi.org/10.1145/1689239.1689243 http://dx.doi.org/10.1518/001872008X312198 iJIM – Vol. 7, No. 2, April 2013 The Implications of Mobile Notifications for User Experience of a Social Network Service