International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 15, 2021 Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions https://doi.org/10.3991/ijim.v15i15.23897 Atilla Wohllebe1(), Manuel Rolf Adler1, Szilárd Podruzsik2 1Hungarian University of Agriculture and Life Sciences, Kaposvár, Hungary 2Institute of Economics, Budapest, Hungary atilla.wohllebe@gmail.com Abstract—With the increasing relevance of mobile apps for companies, push notifications to address app users are also becoming more important. While the acceptance factors of push notifications have already been extensively researched, the effect of the different design elements on user interaction by opening the mobile app is still completely unexplored. Based on existing scientific findings from related fields, especially banner advertising and e-mail marketing, the authors first develop hypotheses on the effect of title, button and image on user interaction with push notifications. In several experiments the hypotheses are tested using the example of a mobile shopping app. The results are evaluated using Chi-square test and Cramer’s V. While the use of a title seems to have a positive effect on interaction rates, the hypotheses on the positive effect of but- tons and images on interaction rates have to be rejected. Keywords—push notifications, mobile apps, mobile push notifications, design elements, cramer’s V 1 Introduction Smartphones and mobile apps are the most important ways globally to access the Internet: More than half of all Internet traffic is generated via mobile devices [1], while at the same time around 90 percent of Europeans are connected to the Internet - and the global trend is still rising [2]. From the user’s point of view, mobile apps are among the most important functionalities of smartphones and now exist for practi- cally all areas of life, such as shopping, communication, travel, education, health or entertainment [3–7]. One of the core functionalities of mobile apps is the so-called push notifications. The notifications, which are displayed on the home screen or the lock screen of a smartphone, regularly provide app users with current content, e.g. offers or news [8]. At the same time, the push notifications also serve to bring existing app users back to the app again. Commercial expectations of mobile apps from companies are high [9]. Especially in the retail sector, mobile apps are becoming increasingly relevant [10]. Push notifications play an important role in meeting these expectations [11]. iJIM ‒ Vol. 15, No. 15, 2021 35 mailto:atilla.wohllebe@gmail.com Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions The impact of push notifications on mobile app usage in the broadest sense has been studied frequently in the past from different perspectives, such as the influence of content or personalization [12], the use of location-based data [13] the influence of frequency [8], [14]. A gap in previous research on push notifications is the use of different design ele- ments that are specifically possible for this type of advertising: for example, a title, an image or a button can be added to the actual text of a push notification [15], [16]. However, there seems to be practically no knowledge about how the design elements specific to this form of advertising influence user engagement with the notifications [17], [18]. In line with the S-O-R model, push notifications from mobile apps can be under- stood as a stimulus that acts on an organism - its recipient - and causes a reaction, for example the opening of an app [19]. Whether and how the organism reacts to the stim- ulus also depends on the attitude the organism, i.e. the recipient, develops towards the message just received [20]. According to the Attitude Toward an Ad Model, this attitude depends not only on the advertiser and the mood of the recipient, but also on the nature of the ad itself [20]. The consumer’s desire for relevant information on the one hand and the curiosity aroused by a lack of information on the other hand are opposed: On the one hand, many findings indicate that relevant, even personalized information is desired [21–23]. On the other hand, the information gap theory suggests that a lack of information could increase curiosity and therefore lead to higher interaction rates [24]. This paper therefore examines the question of how the use of titles, images and buttons as design elements in push notifications affects user engagement with the noti- fications. In the literature review, existing relevant scientific findings on push notifica- tions are presented first. Furthermore, existing literature on the design of banner ads (e.g. [25–28]) and on the design of subject lines in e-mail marketing (e.g. [29–31]) will be reviewed. The hypotheses subsequently formed on this basis are then examined by means of several A/B test experiments, which has already proven to be a successful research method in similar contexts [32]. This is followed by the summary of the findings and the conclusion. 2 Literature review 2.1 Perception and impact of push notifications Push notifications are an instrument of app marketing and can be considered an important driver for app usage [33]. By sending them, app publishers can expect virtually instant attention from their existing user base and corresponding app openings [34]. Con- tent is the decisive factor for user response to the notifications. The literature constantly emphasizes the importance of content relevance from a user’s perspective [35–37]. However, any push notification must also be considered as an interruption and thus carries the risk of disturbing users [38]. Such interferences can lead to frustration and 36 http://www.i-jim.org Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions even to the uninstallation of an app by the user [8], [39]. In this respect, the relevance of the content repeatedly must be emphasized. The perceived usefulness is considered to be the driver of the relevance of push notifications. In the shopping environment, for example, this is driven by coupons, discounts and other price advantages [35], [40]. In addition, the importance of per- sonalized content adapted to the behavior and socio-demographics of the respective consumer is emphasized [41] and also plays a role in the context of mobile apps [42]. A certain entertainment factor can also be seen as a driver for the interaction of app users with push notifications [43]. 2.2 Design of banner ads Since the previous findings on the influence of the design elements of push notifi- cations are only very limited, the effect of the design in banner advertising is also con- sidered, because this advertising channel has similar possibilities, at least in its design. A/B testing is a method that is repeatedly used in the context of banner advertising to examine the effect of different design elements and aspects [32]. Basically, it can be stated that, in addition to the content, which has already been identified as decisive for performance in the context of push notifications, the color design and animation of banners also have an effect on performance in terms of clicks. However, this effect can also vary depending on the context [28]. Larger banners seem to have a clearly positive effect on the click rate [25], [27], [44]. Generally speaking, the amount of information displayed on the banner also has a positive effect on the click rate [44], [45]. Surprisingly, more text on a banner also seems to have a positive effect on user interaction, contrary to the expectations of the authors one study [44]. In the context of search engine advertisements, a study analyz- ing 57 million impressions and 185,000 clicks reveals that click-through rate is posi- tively influenced by brand and price while promotions and questions have a negative effect [45]. The effect of animations is controversial, with some studies identifying them as a positive influencing factor [25], [27] and some as a negative influencing factor [44]. Still other surveys do not find a clearly positive influence [46], [47]. Especially relevant seems to be the picture in the user perception. This is seen by internet users even before the logo and the text on the banner as a study using eye-tracking technology shows [26]. In contrast, there is no homogeneous picture with regard to the use of suggestive prompts for action (often also called “call-to-action” or “action phrase”): Some sources identify these as having a negative effect on consumer interaction [44], while other, albeit much older sources identify them as positive [47]. 2.3 Subject line design in e-mail marketing In addition to the effect of the design in banner advertising, the effect of the design in e-mail marketing is examined in the following. The functionality of the advertising iJIM ‒ Vol. 15, No. 15, 2021 37 Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions channel is similar in that it also addresses consumers who have already come to terms with the company’s brand by registering for the newsletter (in the case of the app: by installing the app). Visually, however, advertising via e-mail newsletters is only comparable to push notifications to a limited extent. In this respect, the literature presented in the following focuses primarily on findings regarding subject lines. Sub- ject lines can be seen as comparable to push notifications as the length of the text is similar. In general, personalized, content-related subject lines are particularly suitable for achieving higher opening rates, as can be determined using A/B testing [30]. Other sources also emphasize the relevance of high information content in straightforward subject lines [29]. The use of emotions is also identified as a positive driver of open rates in email marketing messages based on an A/B test evaluation of an email message sent to 1,409,963 recipients [31]. With regard to the length of subject lines, contradictory findings are repeatedly found in research. It has been established that short subject lines do not have a signifi- cantly better effect than longer ones. Furthermore the research results of the same study indicate a slightly positive effect for custom and content-specific subject lines [30]. In some cases, there is evidence that very short subject lines achieve even worse results than longer ones [29]. Based on the Information Gap Theory [24], other research suggests that an empty subject line should actually achieve better interaction rates because it arouses the curi- osity of the recipient. In fact, this assumption cannot be confirmed by the authors [48]. Nevertheless, other sources, contrary to the results presented above [29], [30], [48], find that short [31] or completely empty subject lines [49] perform better than longer subject lines. These largely contradictory results of different studies show how significant the dif- ferences between theoretical and practical or experimental findings are [50]. 2.4 Development of hypotheses Regarding the effect of title, button and image, one hypothesis is made for each design element. Since a title allows to display more information with a push notification and thus the advertisement appears larger [25–27], [44], [45] it is assumed that H1: A title has a positive effect on the probability that users tap a push notification. With regard to the use of a button, a positive effect on interaction rates is also assumed, since the button makes the push notification appear larger and acts as a call- to-action [25], [27], [44], [47]. It is therefore stated that H2: A button has a positive effect on the probability that users tap a push notification. A picture also has a positive effect on the size of the push notification as an adver- tisement, especially since it plays a particularly important role in the user’s perception process [25–27], [44]. A positive effect is also assumed here: H3: An image has a positive effect on the probability that users tap a push notification. Figure 1 shows the different options to include the respective design elements in a push notification. 38 http://www.i-jim.org Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions Fig. 1. Push notification design elements 3 Empirical analysis 3.1 Experimental setup and data collection The established hypotheses on the effect of title, image and button on the perfor- mance of push notifications are to be tested by using A/B Testing. This method is used regularly in similar experimental setups [30], [51]. Two push notifications with only one element differing are sent to two different, randomly selected target groups. By using the Chi-square test, the extent to which the observed frequencies differ from each other is examined. The null hypothesis assumes that the variables are statistically independent of each other [52]. Furthermore, the strength of the correlation is also mea- sured by means of Cramer’s V [53]. A value of 0.1 is considered weak, 0.3 as medium and a value of 0.5 or greater as strong correlation [54], [55]. To increase the reliability of the results, the effect of each design element is mea- sured not only on one message, but on five messages. Thus, a total of five messages are sent three times in two variants each, i.e. 30 messages in total. The push notifications are sent per variant to 7,500 users of a mobile shopping app. Success is measured in each case by tapping the notification. 3.2 Statistical analysis and results For each hypothesis it is shown per message how many push interactions were generated without the tested element and with the tested element. In addition, the Chi-square value and Cramer’s V-value and the interpretation with respect to the hypoth- esis are shown. The following applies to all data: Each variant of each message was sent to a total of 7,500 users. Each Chi-square test is performed with c2crit = 3.84, df = 1, (1-α) = .95. iJIM ‒ Vol. 15, No. 15, 2021 39 Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions Table 1 shows the results of the five A/B tests in which push notifications without and with titles were sent to 7,500 recipients each. In all cases, the Chi-square value determined is above the critical value, so that in all cases there is a significant differ- ence. H1 is accordingly regarded as confirmed. Using Cramer’s V, the strength of the effect is determined. In all cases, this is below 0.1000, so that, despite confirmation of the hypothesis, only a weakly positive effect of a title on the openings can be assumed. Table 2 shows the results of the A/B tests regarding the button. In three out of five cases the openings with buttons are higher than those without buttons, but at the same time no significant difference can be detected. In two cases the openings without but- ton are even higher than the openings with button. In one case this difference is even significant. H2 is rejected completely, but at least no negative effect of a button on the interaction rates can be assumed. The results for the effect of an image as a design element of push notifications are shown in Table 3. While H3 is rejected in three cases, it can be confirmed in two cases. In these two cases, however, Cramer’s V is only in the weak range. Table 1. Effect of titles in push notifications – experimental results Msg ID Opens w/o Opens w/ Chi Square Cramer’s V Hypothesis 1 996 1,168 25.70 .0550 Confirm 2 586 657 7.91 .0313 Confirm 3 760 873 15.05 .0427 Confirm 4 1,102 1,330 40.07 .0682 Confirm 5 523 604 11.61 .0380 Confirm Table 2. Effect of buttons in push notifications – experimental results Msg ID Opens w/o Opens w/ Chi Square Cramer’s V Hypothesis 1 766 798 1.21 .0121 Reject 2 445 445 – – Reject 3 713 733 0.51 .0079 Reject 4 604 554 3.85 .0218 Reject 5 677 656 0.60 .0086 Reject Table 3. Effect of images in push notifications – experimental results Msg ID Opens w/o Opens w/ Chi Square Cramer’s V Hypothesis 1 678 647 1.30 .0126 Reject 2 453 434 0.75 .0097 Reject 3 547 558 0.21 .0051 Reject 4 322 362 4.74 .0246 Confirm 5 267 304 4.93 .0252 Confirm 40 http://www.i-jim.org Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions In summary, H3 is rejected accordingly. No clear evidence can be found that the presence of an image has a clearly positive effect on interaction rates. It must be noted that only the presence of an image was tested. Results may differ based on which spe- cific image is used. 4 Summary and discussion After reviewing existing research results on the effect of push notification design, related literature from the fields of banner advertising and e-mail marketing was con- sidered. The developed hypotheses assume that the design elements title, button and image have a positive effect on the interaction rates. The hypotheses were empirically tested in several experiments. Only for the title such a positive correlation could be shown, whereas the strength of the effect is only weakly positive. Table 4 summarizes the hypotheses and their results. The results make it clear that the already mentioned, partly significant differences between theoretical derivation and practical or experimental experience in e-mail mar- keting are also likely to be present in the area of mobile push notifications [50]. With regard to limitations, it must be noted that the results may vary depending on the context, e.g. shopping, news or messaging [28]. The results of this paper are there- fore mainly related to mobile shopping apps. The metric is also a limitation in that the results can only be related to direct opening of push notifications, but not to indirect opening of push notifications [36]. Probably the most important limitation is that here only the presence of the mentioned elements was tested, but not the concrete design. It would be conceivable that there are differences in the effect on the probability of inter- action, depending on how a title is concretely formulated or which picture is concretely chosen. 5 Conclusion and further research This research fills an important research gap in the investigation of the effect of push notification with regard to the different design elements. Similar issues have already been extensively researched in other marketing channels and have been highlighted accordingly in this paper. The results of the experiments carried out here to test the hypotheses put forward have a high practical relevance for marketing via mobile shopping apps. While a title Table 4. Summarized results of hypothesis testing Hypothesis Results H1: A title has a positive effect on the probability that users tap a push notification. Confirmed H2: A button has a positive effect on the probability that users tap a push notification. Rejected H3: An image has a positive effect on the probability that users tap a push notification. Rejected iJIM ‒ Vol. 15, No. 15, 2021 41 Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions obviously has a positive effect on the probability of interaction, marketers should criti- cally examine the effort required to create images for push notifications and also ques- tion the use of buttons. This paper focuses exclusively on the existence of the different design elements. For further research, the authors propose to examine in particular how concrete design variants of the different design elements affect interaction rates. It is conceivable, for example, that titles have different effects depending on their length, tonality, or content. 6 References [1] StatCounter, “Anteil mobiler Endgeräte an allen Seitenaufrufen nach Regionen weltweit im Jahr 2018,” Statista, 2019. https://de.statista.com/statistik/daten/studie/217457/umfrage/ anteil-mobiler-endgeraete-an-allen-seitenaufrufen-weltweit (accessed Dec. 31, 2019). [2] Eurostat, “Anteil der Internetnutzer in der Europäischen Union (EU-28) nach Ländern im Jahr 2018,” Statista, 2018. https://de.statista.com/statistik/daten/studie/184636/umfrage/ internetreichweite-anteil-der-nutzer-in-europa (accessed Dec. 31, 2019). [3] H.-J. Kim and J.-Y. Rha, “Predicting the Drivers of the Intention to Use Mobile Learning in South Korea,” International Journal of Interactive Mobile Technologies (iJIM), vol. 12, no. 1, Art. no. 1, Jan. 2018, https://www.doi.org/10.3991/ijim.v12i1.7688. [4] S. Papadakis and M. Kalogiannakis, Eds., Mobile Learning Applications in Early Childhood Education: IGI Global, 2020. https://doi.org/10.4018/978-1-7998-1486-3 [5] A. Wohllebe, F. Ross, and S. Podruzsik, “Influence of the Net Promoter Score of Retailers on the Willingness of Consumers to Install Their Mobile App,” Int. J. Interact. Mob. Technol., vol. 14, no. 19, 2020, https://www.doi.org/10.3991/ijim.v14i19.17027. [6] M. Kalogiannakis and S. Papadakis, “Combining mobile technologies in environmen- tal education: a Greek case study,” IJMLO, vol. 11, no. 2, p. 108, 2017, https://www.doi. org/10.1504/IJMLO.2017.084272. [7] F. Ross, “Hearing Aid Accompanying Smartphone Apps in Hearing Healthcare. A System- atic Review,” Applied Medical Informatics, vol. 42, no. 4, Nov. 2020, [Online]. Available: https://ami.info.umfcluj.ro/index.php/AMI/article/view/792. [8] A. Wohllebe, “Consumer Acceptance of App Push Notifications: Systematic Review on the Influence of Frequency,” International Journal of Interactive Mobile Technologies (iJIM), vol. 14, no. 13, 2020, https://www.doi.org/10.3991/ijim.v14i13.14563. [9] BITKOM, “Umsatz mit mobilen Apps in Deutschland in den Jahren 2008 bis 2018 sowie eine Prognose für 2019 (in Millionen Euro),” Statista, 2019. https://de.statista.com/statis- tik/daten/studie/173810/umfrage/umsatz-mit-mobilen-apps-in-deutschland-seit-2009 (accessed Oct. 19, 2019). [10] R. Deckert and A. Wohllebe, Digitalisierung und Einzelhandel: Taktiken und Technologien, Praxisbeispiele und Herausforderungen, 1st ed. Wiesbaden, Germany: Springer Gabler, 2021. https://doi.org/10.1007/978-3-658-33090-3 [11] X.-L. Pham, T.-H. Nguyen, W.-Y. Hwang, and G.-D. Chen, “Effects of Push Notifications on Learner Engagement in a Mobile Learning App,” in 2016 IEEE 16th International Confer- ence on Advanced Learning Technologies (ICALT), Austin, TX, USA, Jul. 2016, pp. 90–94, https://www.doi.org/10.1109/ICALT.2016.50. [12] X. Wang, Z. Hong, Y. C. Xu, C. Zhang, and H. Ling, “Relevance judgments of mobile com- mercial information: Relevance Judgment of Mobile Commercial Information,” J Assn Inf Sci Tec, vol. 65, no. 7, pp. 1335–1348, Jul. 2014, https://www.doi.org/10.1002/asi.23060. 42 http://www.i-jim.org https://doi.org/10.4018/978-1-7998-1486-3 https://doi.org/10.1007/978-3-658-33090-3 Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions [13] N. Allurwar, B. Nawale, and S. Patel, “Beacon for Proximity Target Marketing,” Interna- tional Journal of Engineering And Computer Science, vol. 5, no. 5, pp. 16359–16364, 2016, https://www.doi.org/10.18535/IJECS/V5I5.08. [14] J. Freyne, J. Yin, E. Brindal, G. A. Hendrie, S. Berkovsky, and M. Noakes, “Push Notifi- cations in Diet Apps: Influencing Engagement Times and Tasks,” International Journal of Human–Computer Interaction, vol. 33, no. 10, pp. 833–845, Oct. 2017, https://www.doi.org /10.1080/10447318.2017.1289725. [15] Google Firebase, “Notification Config,” Firebase Documentation, 2020. https://firebase.google. com/docs/reference/fcm/rest/v1/projects.messages#Notification (accessed Sep. 03, 2020). [16] Airship Inc., “Optional Features,” Airship Documentation, 2020. https://docs.airship.com/ reference/messages/optional-features/ (accessed Sep. 03, 2020). [17] Google Scholar, “‘Push Notification Design’ at Google Scholar,” Google Scholar, Sep. 02, 2020. https://scholar.google.com/scholar?hl=de&as_sdt=0%2C5&q=push+notification+ design&btnG= (accessed Sep. 02, 2020). [18] ResearchGate, “‘Push Notifications Design’ at ResearchGate,” ResearchGate, Sep. 02, 2020. https://www.researchgate.net/search.Search.html?type=researcher&query=push%20 notification%20design (accessed Sep. 02, 2020). [19] R. S. Woodworth, Psychology: A study of mental life. New York: Henry Holt and Co, 1921. https://doi.org/10.1037/13976-000 [20] S. B. MacKenzie and R. J. Lutz, “An Empirical Examination of the Structural Anteced- ents of Attitude toward the Ad in an Advertising Pretesting Context,” Journal of Marketing, vol. 53, no. 2, p. 48, Apr. 1989, https://www.doi.org/10.2307/1251413. [21] A. P. Kapoor and M. Vij, “How to Boost your App Store Rating? An Empirical Assessment of Ratings for Mobile Banking Apps,” J. theor. appl. electron. commer. res., vol. 15, no. 1, 2020, https://www.doi.org/10.4067/S0718-18762020000100108. [22] A. Kazeminia, M. Kaedi, and B. Ganji, “Personality-Based Personalization of Online Store Features Using Genetic Programming: Analysis and Experiment,” J. theor. appl. electron. commer. res., vol. 14, no. 1, Jan. 2019, https://www.doi.org/10.4067/S0718- 18762019000100103. [23] A. Mehrotra, V. Pejovic, J. Vermeulen, R. Hendley, and M. Musolesi, “My Phone and Me: Understanding People’s Receptivity to Mobile Notifications,” in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, California, USA, May 2016, pp. 1021–1032, https://www.doi.org/10.1145/2858036.2858566. [24] G. Loewenstein, “The psychology of curiosity: A review and reinterpretation,” Psychological Bulletin, vol. 116, no. 1, pp. 75–98, 1994, https://www.doi.org/10.1037/0033-2909.116.1.75. [25] C.-H. Cho, “The Effectiveness of Banner Advertisements: Involvement and Click-through,” Journalism & Mass Communication Quarterly, vol. 80, no. 3, pp. 623–645, Sep. 2003, https://www.doi.org/10.1177/107769900308000309. [26] M. Köster, M. Rüth, K.-C. Hamborg, and K. Kaspar, “Effects of personalized banner ads on visual attention and recognition memory,” Applied Cognitive Psychology, vol. 29, no. 2, pp. 181–192, 2015. https://doi.org/10.1002/acp.3080 [27] H. Li and J. L. Bukovac, “Cognitive impact of banner ad characteristics: An experimental study,” Journalism & Mass Communication Quarterly, vol. 76, no. 2, pp. 341–353, 1999. https://doi.org/10.1177/107769909907600211 [28] R. Lothia, N. Donthu, and E. K. Hershberger, “The impact of content and design elements on banner advertising click-through rates,” Journal of advertising Research, vol. 43, no. 4, pp. 410–418, 2003. https://doi.org/10.2501/JAR-43-4-410-418 [29] P. S. Brenner, C. Cosenza, and F. J. Fowler Jr, “Which Subject Lines and Messages Improve Response to E-mail Invitations to Web Surveys?,” Field Methods, 2020. https://doi. org/10.1177/1525822X20929647 iJIM ‒ Vol. 15, No. 15, 2021 43 https://doi.org/10.1037/13976-000 https://doi.org/10.1002/acp.3080 https://doi.org/10.1177/107769909907600211 https://doi.org/10.2501/JAR-43-4-410-418 https://doi.org/10.1177/1525822X20929647 https://doi.org/10.1177/1525822X20929647 Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions [30] M. O’Connell, “Enhancing the effectiveness of e-mail newsletters: how content (subject lines, intros) impacts response rates,” Hussman School of Journalism and Media, Chapel Hill, North Carolina, 2008. [31] S. Stupar-Rutenfrans, D. de Koff, and J. van den Elst, “The Effect of Subject Lines on Open Rates of Email Marketing Messages,” Advances in Social Sciences Research Journal, vol. 6, no. 7, pp. 181–188, 2019. https://doi.org/10.14738/assrj.67.6749 [32] A. Sigel, G. Braun, and M. Sena, “The impact of banner ad styles on interaction and click- through rates,” Issues in Information Systems, vol. 9, no. 2, pp. 337–342, 2008. [33] A. Wohllebe and M. Hillmers, “Towards a Scientific Definition of App Marketing – A Practice-Oriented Approach Using Scientific and Grey Literature,” IJARBM, vol. 2, no. 1, pp. 13–25, Jun. 2021, https://www.doi.org/10.51137/ijarbm.2021.2.1.2. [34] A. Sahami Shirazi, N. Henze, T. Dingler, M. Pielot, D. Weber, and A. Schmidt, “Large- scale assessment of mobile notifications,” in Proceedings of the 32nd annual ACM confer- ence on Human factors in computing systems - CHI ’14, Toronto, Ontario, Canada, 2014, pp. 3055–3064, https://www.doi.org/10.1145/2556288.2557189. [35] D. Ý. Rigollet and H. Kumlin, “Consumer Attitudes towards Push Notifications: As a Mar- keting Tool to Trigger Impulse Buying Behaviour in Smartphone Users,” 2015. [36] D. Ahrholdt, G. Greve, and G. Hopf, Online-Marketing-Intelligence: Kennzahlen, Erfolgs- faktoren und Steuerungskonzepte im Online-Marketing. Wiesbaden: Springer Fachmedien Wiesbaden, 2019. https://doi.org/10.1007/978-3-658-26562-5 [37] B. Berman, “Planning and implementing effective mobile marketing programs,” Busi- ness Horizons, vol. 59, no. 4, pp. 431–439, Jul. 2016, https://www.doi.org/10.1016/ j.bushor.2016.03.006. [38] S. T. Iqbal and E. Horvitz, “Disruption and recovery of computing tasks: field study, analy- sis, and directions,” in Proceedings of the SIGCHI Conference on Human Factors in Com- puting Systems - CHI ’07, San Jose, California, USA, 2007, pp. 677–686, https://www.doi. org/10.1145/1240624.1240730. [39] T. Westermann, S. Möller, and I. Wechsung, “Assessing the Relationship between Tech- nical Affinity, Stress and Notifications on Smartphones,” in Proceedings of the 17th Inter- national Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct - MobileHCI ’15, Copenhagen, Denmark, 2015, pp. 652–659, https://www.doi. org/10.1145/2786567.2793684. [40] M. Adler and A. Wohllebe, “Consumers Choosing Retailers On Online Marketplaces: How Can Retailers Differentiate Apart From The Price? – An Exploratory Investigation,” IJARBM, vol. 1, no. 1, pp. 27–36, Dec. 2020, https://www.doi.org/10.51137/ijarbm.2020.1.1.3. [41] S. Kim, T. H. Baek, Y.-K. Kim, and K. Yoo, “Factors affecting stickiness and word of mouth in mobile applications,” Jnl of Res in Interact Mrkting, vol. 10, no. 3, pp. 177–192, Aug. 2016, https://www.doi.org/10.1108/JRIM-06-2015-0046. [42] S. Papadakis, N. Zaranis, and M. Kalogiannakis, “Parental involvement and attitudes towards young Greek children’s mobile usage,” International Journal of Child-Computer Interac- tion, vol. 22, p. 100144, Dec. 2019, https://www.doi.org/10.1016/j.ijcci.2019.100144. [43] K. Jacob and D. Gupta, “Factors influencing people to later visit mobile app based on push notifications: A comparison between formats.” 2017, [Online]. Available: https://www.amrita.edu/sites/default/files/factors-influencing-people-to-later-visit-mo- bile-app-based-on-push-notifications-a-comparison-between-formats.pdf. [44] H. Robinson, A. Wysocka, and C. Hand, “Internet advertising effectiveness: the effect of design on click-through rates for banner ads,” International Journal of Advertising, vol. 26, no. 4, pp. 527–541, 2007. https://doi.org/10.1080/02650487.2007.11073031 [45] G. Atkinson, C. Driesener, and D. Corkindale, “Search engine advertisement design effects on click-through rates,” Journal of Interactive Advertising, vol. 14, no. 1, pp. 24–30, 2014. https://doi.org/10.1080/15252019.2014.890394 44 http://www.i-jim.org https://doi.org/10.14738/assrj.67.6749 https://doi.org/10.1007/978-3-658-26562-5 https://doi.org/10.1080/02650487.2007.11073031 https://doi.org/10.1080/15252019.2014.890394 Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions [46] S. Zorn, D. Olaru, T. Veheim, S. Zhao, and J. Murphy, “Impact of animation and language on banner click-through rates,” Journal of Electronic Commerce Research, vol. 13, no. 2, pp. 173–183, 2012. [47] N. Rae and M. Brennan, “The relative effectiveness of sound and animation in web ban- ner advertisments,” Marketing bulletin-department of marketing massey university, vol. 9, pp. 76–82, 1998. [48] N. Sappleton and F. Lourenço, “Email subject lines and response rates to invitations to par- ticipate in a web survey and a face-to-face interview: the sound of silence,” International Journal of Social Research Methodology, vol. 19, no. 5, pp. 611–622, 2016. https://doi.org/ 10.1080/13645579.2015.1078596 [49] S. R. Porter and M. E. Whitcomb, “E-mail subject lines and their effect on web survey viewing and response,” Social Science Computer Review, vol. 23, no. 3, pp. 380–387, 2005. https://doi.org/10.1177/0894439305275912 [50] C. Teiu, “Email Subject Lines Analysis for High Open Rates in Email Marketing,” in BASIQ International Conference on New Trends in Sustainable Business and Consump- tion, Messina, Italy, 2020, pp. 835–840, [Online]. Available: https://www.researchgate. net/profile/Bassel_Diab/publication/342124082_BASIQ_2020_Conference_proceedings/ links/5ee37056458515814a583fe1/BASIQ-2020-Conference-proceedings.pdf#page=835. [51] A. Wohllebe, T. Stoyke, and S. Podruzsik, “Incentives on E-Commerce App Downloads in Medium Apps: A Case Study on the Effects of Coupons and Bonus Points,” Int. J. Interact. Mob. Technol., vol. 14, no. 19, 2020, https://www.doi.org/10.3991/ijim.v14i19.16427. [52] U. Kuckartz, S. Rädiker, T. Ebert, and J. Schehl, “Kreuztabelle, Chi-Quadrat und Zusammenhangsmaße,” in Statistik: Eine verständliche Einführung, U. Kuckartz, S. Rädiker, T. Ebert, and J. Schehl, Eds. Wiesbaden: VS Verlag für Sozialwissenschaften, 2010, pp. 81–102. https://doi.org/10.1007/978-3-531-92033-7_4 [53] A. C. Acock and G. R. Stavig, “A Measure of Association for Nonparametric Statistics,” Social Forces, vol. 57, no. 4, p. 1381, Jun. 1979, https://www.doi.org/10.2307/2577276. [54] J. Cohen, Statistical Power Analysis for the Behavioral Sciences. Elsevier, 1977. [55] D. Keller, “Effektstärke,” Statistik und Beratung - Daniela Keller, Jul. 28, 2015. https:// statistik-und-beratung.de/2015/07/effektstaerke/ (accessed Apr. 26, 2020). 7 Authors Atilla Wohllebe is a consultant, employee and researcher. Specializing in digital CRM and retail digitalization, he holds a B. A. in Business Administration and an M. Sc. in E-Commerce. He is a member of the board of the Digital Analytics Asso- ciation Germany (DAA) and a member of the International Association of Online Engineering (IAOE). He is currently pursuing a PhD program at MATE Hungar- ian University of Agriculture and Life Sciences - Kaposvár Campus, Kaposvár, Hungary. Manuel Rolf Adler holds a B. Sc. in International Business Administration and a LL. M. in Business Law. Currently, he is a PhD student at the Doctoral School in Man- agement and Organizational Sciences, Hungarian University of Agriculture and Life Sciences - Kaposvár Campus, Hungary. His primary field of research is the quantifica- tion of regulatory risk and the impact of prudential regulation on financial institutions’ business models. M. R. Adler has several years of work experience in the German financial and banking sector, especially in regulation and compliance. iJIM ‒ Vol. 15, No. 15, 2021 45 https://doi.org/10.1080/13645579.2015.1078596 https://doi.org/10.1080/13645579.2015.1078596 https://doi.org/10.1177/0894439305275912 https://doi.org/10.1007/978-3-531-92033-7_4 Paper—Influence of Design Elements of Mobile Push Notifications on Mobile App User Interactions Szilárd Podruzsik is a researcher working at the Centre for Economic and Regional Studies, Institute of Economics, Budapest, Hungary. He also works as a supervisor at Hungarian University of Agriculture and Life Sciences. He holds a PhD from the Budapest University of Economics. His research focus is on small and medium-sized enterprises, food production, consumption and their determinants. He has already pub- lished over eighty papers as an author or co-author and participated in several research projects as a leader or participant. Article submitted 2021-05-11. Resubmitted 2021-06-08. Final acceptance 2021-06-08. Final version published as submitted by the authors. 46 http://www.i-jim.org