Paper title (Paper Title style) Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 1 Elucidating the role of emotion in privacy-concerns: A text-Convolutional Neural Network (text-CNN)-based tweets analysis of contact tracing apps Mihir Mehta Indian Institute of Management Raipur, India Sourya Joyee De Indian Institute of Management Raipur, India souryajoyee@iimraipur.ac.in Manojit Chattopadhyay Indian Institute of Management Raipur, India Abstract The extant contact tracing privacy literature is yet to explore the significance of user emotions in privacy-related decision-making such as whether to use such potentially privacy-invasive apps. Using social media analytics, the present study examines users’ privacy-related emotions stimulated by privacy-related aspects of contact tracing apps. A text-Convolutional Neural Network (Text-CNN)-based emotion analysis of tweets on the Indian contact tracing app Aarogya Setu and its Singaporean counterpart TraceTogether conducted in the paper reveals that users expressed negative privacy-related emotions towards these apps indicating high levels of perceived privacy risks and the perceived lack of privacy protection. For TraceTogether, users have also exhibited positive emotions to appreciate the steps taken by the government to protect their privacy. Based on these findings, the government/data controllers can devise strategies to assuage users’ negative emotions and promote positive emotions to encourage the adoption of contact tracing apps. This work incorporates privacy related emotions as key informants about user privacy concerns within the Privacy Calculus Theory. By relying on candid user opinions available through rich but inexpensive user- generated content, the research provides a quick, reliable, and cost-effective approach to study potential app users’ emotions to gain insights into privacy concerns related to any e- governance platform. Keywords: Privacy Calculus Theory, text-Convolutional Neural Network, emotion analysis of tweets, contact tracing apps, perceived privacy risks, perceived privacy protections. 1 Introduction With the rapid worldwide spread of COVID-19, digital contact tracing mobile applications have been rolled out in many countries to automatically track locations and movements of infected individuals and their social contacts with high levels of precision, quickly and efficiently (Cho, Ippolito, & Yu, 2020). Privacy researchers as well as users have expressed deep concerns regarding privacy risks resulting from the collection, processing, and sharing of personal data by these apps and the lack of suitable privacy protection measures (Baumgärtner et al., 2020; Rowe, 2020; Fahey & Hino, 2020; Georgieva et al., 2021). Profiling and tracking of specific users (Wen et al., 2020; Bengio et al., 2021; Leith & Farrell, 2020), de- anonymization of infected persons leading to stigmatization (Bäumgartner et al., 2020), data misuse by third parties (Azad et al. 2020) and government surveillance (Rowe, 2020; Fahey Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 2 & Hino, 2020; Georgieva et al., 2021) have been identified as some of the major privacy risks associated with contact tracing apps. The success of contact tracing applications is largely dependent on mass citizen acceptance (Fox et al., 2021) and with greater level of privacy awareness, convincing citizens to install and use these applications presents a significant challenge (Fahey & Hino, 2020). Emotion has been shown to have a significant impact on decision-making (Lerner et al., 2015; Zhang & Xu, 2016). However, the existing privacy literature on contact tracing is yet to extensively consider privacy emotion, such as anger at privacy risks like surveillance and profiling and happiness at good privacy protection measures such as data protection regulations, as a prime element in privacy-related decision-making such as whether to use the app. Privacy Calculus Theory suggests that a person's intention to disclose personal information is based on a balancing act in which potentially competing factors such as negative beliefs or privacy concerns are weighed against positive beliefs or benefits gained from information disclosure (Culnan & Armstrong, 1999; Masur & Scharkow, 2016). This paper is an early attempt to examine users’ privacy related emotions in the context of contact tracing apps, incorporating the role of privacy emotions as a key informant about user privacy concerns in the Privacy Calculus Theory. Negative privacy emotions can indicate higher privacy risk perception and less appreciation of existing protection measures, leading to hesitation or low willingness to rely on the app (Barnard et al., 2014). Positive privacy emotions, on the other hand, can indicate low privacy risk perception and higher appreciation of existing protection measures, leading to high readiness or willingness to rely on the app (Li et al., 2008). Prior works have explored how privacy concerns affect the willingness of citizens to adopt contact tracing apps in different countries such as Ireland, France, Australia, and Belgium (Fox et al., 2021; Chan and Saqib, 2021; Walrave et al., 2020; Horvath et al., 2020). These studies rely on user surveys for data collection and analysis. A limited number of studies have also been conducted to understand citizens’ opinion on contact tracing in different countries through the analysis of user-generated content (UGC) on social media such as Twitter. While in Ireland, citizens exhibited mostly positive opinion (Rekanar et al., 2021), in some other countries such as India and Brazil, citizens exhibited more negative opinions (Praveen et al., 2020a; Praveen et al., 2021). Privacy concerns have been identified as one of the major reasons behind negative feelings (Praveen et al., 2020a; Praveen et al., 2021; Crable & Sena, 2020). Recent works in the privacy literature (Fiesler & Hallinan, 2018; Gonzalez et al., 2019a, 2019b) have also highlighted the limitations of survey-based methods in privacy and have begun to explore the potential of UGC on SM platforms such as Facebook and Twitter to reveal users’ privacy attitudes and opinions “in the wild”. Our work incorporates privacy-related emotions as key informants of privacy concerns within the Privacy Calculus Theory in the context of potentially privacy-invasive contact tracing apps. The study unveils that negative emotions are related to higher perceived privacy risks and the perceived lack of privacy protection measures, and these emotions are often the result of privacy-related stimuli around the app such as negative reports from news media, statements of privacy experts, ethical hackers, and activists. Similarly, users exhibit positive emotions when they feel that the government has taken sufficient steps to protect their privacy, also a privacy-related stimulus. Based on these findings, the government could devise suitable strategies to assuage users’ negative feelings and promote positive emotions Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 3 to encourage the adoption of contact tracing apps. In contrast to traditional survey-based approaches, the methodological contribution of this research constitutes a quick, reliable, and cost-effective social media analytics-based approach to study potential app users’ emotions to gain insights into privacy concerns related to any e-governance technology or platform. In addition, the study reveals differences in user emotions around certain aspects across different contact tracing apps which prompts further future investigation into why such differences exist. In light of the above discussion, the following research questions are examined by this paper: • How can user-generated content (UGC) on social media be mined for insights into users’ perceived privacy risks and perceived privacy protections for contact tracing apps? • How do emotions on perceived privacy risks and perceived privacy protections for the Indian and Singaporean contact tracing apps differ? The rest of the paper is organized as follows. The first section introduced the paper. The second section discusses the theoretical background of this research. The third section describes the method adopted to analyse Twitter data to gain citizen privacy insights. The fourth section highlights important findings. Subsequently, the fifth section summarizes the theoretical and practical implications of the paper. In the final section, we conclude the paper with directions for future research. 2 Theoretical Background and Review of Literature In the face of a deadly pandemic such as COVID-19, intuition dictates that people would be more concerned about their health leading to an increased willingness to adopt contact tracing apps, even at the potential cost of their privacy. However, experiments conducted across France, Australia and the United States show that salient COVID-19 concerns decrease the intentions of using contact tracing apps due to increased privacy concerns (Chan & Saqib, 2021). A longitudinal, survey-based examination of the competing influences of positive beliefs and privacy concerns on citizens’ acceptance of a contact tracing app shows that citizens’ privacy concerns demonstrate a negative influence on the willingness to rely on the application (Fox et al., 2021). Privacy concerns, informed by users’ perceived privacy risks and perceived privacy protection, relate to negative beliefs and users balance such beliefs with positive ones to decide whether to disclose their personal data, as suggested by the Privacy Calculus Theory (PCT). Emotions have been found to play a key role in privacy-related decision making and users’ emotional responses to privacy aspects of contact tracing apps can be looked upon as key drivers of negative beliefs. Therefore, in this study, we explore how privacy-related emotions can be extracted from user generated content (UGC) on social media. In the rest of this section, we discuss the theoretical background of our work and related prior works. 2.1 Emotions, Decision-making and Privacy Emotions have been defined as “felt tendency toward anything intuitively appraised as good (beneficial) or away from anything intuitively appraised as bad (harmful)” (Arnold, 1960). Emotions are intense, short-lived, and highly conscious affective states (Frijda, 1994) and are relational or directed at a particular object (Smith & Kirby, 2000). Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 4 Emotions have been assumed to be the dominant driver of meaningful decisions (Ekman, 2007; Keltner et al., 2014; Keltner & Lerner 2010; Loewenstein et al., 2001). Integral emotions or emotions arising from the judgment or choice at hand strongly and routinely shape decision making (Damasio, 1994; Greene & Haidt, 2002). The emotion-imbued choice (EIC) model that account for both traditional (rational choice) inputs and emotional inputs depicts that the characteristics of the choice options as well as predicted emotions can influence emotions felt at the time of decision, ultimately impacting the decision (Lerner et al., 2015). The affect-as-information model also explains how emotions may serve as affective feedback that guides people’s judgement, information processing and decisions (Clore et al., 2001). Feelings and emotions can be drivers of privacy decisions (Kehr et al., 2013; Li et al., 2011; Wakefield, 2013; Kehr et al., 2015). Emotions of joy and fear, formed from website interactions, can significantly influence online consumers’ privacy protection beliefs and perceived privacy risks (Li et al., 2011). The effect of the creepiness emotion which is a mixture of fear, anxiety, and strangeness, towards privacy harms associated with new technological features have also been discussed in the privacy literature (Zhang & Xu, 2016). People may find online behavioural targeting to be creepy (Ur et al., 2012) which can discourage online consumers’ purchase intention (Barnard et al., 2014). In the context of healthcare, Anderson and Agarwal (2011) show that emotions linked to one’s health condition play a significant role in the willingness to disclose health information. The interaction with a contact tracing app represents a novel privacy situation where users lack complete knowledge. In such a situation, they cannot rely only on cognition to evaluate the privacy situation and instead, as postulated by the feeling-as-information theory, emotions could provide important feedback about the app’s privacy character, further influencing the decision on whether to use the app (Li et al., 2017). We explore users’ positive and negative emotions stimulated by privacy-related aspects of contact tracing apps. These stimuli may include the privacy features of the app itself (such as privacy policy, permissions, etc.) and interviews, news and/or other publications regarding the app’s privacy (such as discussions by privacy experts, news on regulatory measures on protection of data collected by the app, etc.). Negative privacy emotions may be associated with a higher privacy risk perception and less appreciation of existing protection measures for the app, discouraging its use. Positive privacy emotions, on the other hand, can indicate low privacy risk perception and higher appreciation of existing protection measures, encouraging the use of the app. 2.2 Overview on Privacy Calculus Theory The Privacy Calculus Theory (PCT) has been widely used to enhance the understanding of how users evaluate the fairness of disclosing personal information (Keith et al., 2013; Sun et al., 2015, Xu et al., 2011; Gutierrez et al., 2019). It suggests that a person's intention to disclose personal information is based on a calculus of behaviour, referred to as privacy calculus, in which potentially competing factors are weighed in the light of possible outcomes (Dinevet al., 2008). It assumes that a rational decision-making process involves a cost-benefit analysis where privacy risks represent costs and the transaction itself gives rise to benefits and the resulting calculus determines willingness to disclose information (Li, 2012; Wigan, 2020; Wildenauer, 2020). Within this theory, factors such as perceived risks and vulnerability, computer anxiety and previous experience with privacy invasion have been shown to raise privacy concerns and discourage information disclosure whereas factors such as website Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 5 reputation and vendor interventions such as privacy policies have been shown to mitigate privacy concerns and encourage information disclosure (Li, 2012). In the context of healthcare, prior research has extended the privacy calculus to explicitly incorporate health status related emotion as a key driver of health information disclosure decision (Anderson & Agarwal, 2011). Kehr et al. (2015) explored the role of an individual’s current mood, induced before the actual decisive situation occurred on privacy-related risk perceptions within a situational privacy calculus. While the benefits of contact tracing apps have been well-advertised by governments and the World Health Organization (WHO), privacy risks and measures taken to protect users from these risks remain much less discussed. Therefore, taking the negative beliefs of privacy concerns as the base of our paper, we examine citizens’ candid expressions of emotions related to their perceived privacy risks and protections of contact tracing apps on the social media platform Twitter. We thus look upon privacy emotions stimulated by privacy aspects of a technology as a key informant of user privacy concerns in Privacy Calculus Theory, within the context of contact tracing apps. 2.3 Overview on Perceived privacy risk Perceived privacy risk has been defined as the fear of the potential losses that would be incurred if personal information is disclosed without permission (Featherman & Pavlou, 2003) and the perceived risk of opportunistic behaviour related to the disclosure of personal information (Dinev & Hart, 2006). Consistent with expectancy theory’s explanation that individuals are motivated to minimize negative outcomes, perceived privacy risk has been shown to negatively influence intention to disclose personal information (Malhotra, Kim, and Agarwal, 2004). Users of contact tracing apps fear the emergence of a surveillance state and the creation of detailed profiles that could lead to discrimination and stigmatization (Praveen et al., 2020b; Rowe, 2020; Fahey & Hino et al., 2020; Georgieva et al., 2021). Perceived privacy risks for non-commercial transactions, especially those within e-government, have received very less attention till date. Risk perceptions can negatively affect user’s inclination to use e-government services (Beldad et al., 2011). We represent perceived privacy risk through users’ negative emotions of fear and/or anger towards negative consequences due to opportunistic behaviour of the government. 2.4 Overview on perceived privacy protection Perceived privacy protection refers to users’ perception of the likelihood that the service provider with whom they share their personal data will adopt enough measures to protect such data from misuse such as undesirable disclosure or unauthorized use (Kim et al., 2009; Shaw & Sergueeva, 2019). With increasing awareness, users are concerned about their privacy protection. They expect service providers to protect them from any potential risks, loss, or fraud (Featherman et al., 2010). We represent perceived privacy protection as citizens’ perception of the effectiveness of the privacy protection measures being used by contact tracing apps. They may express positive emotion of happiness when they feel satisfied with a certain protection measure and may similarly express negative emotions of anger or fear when they feel dissatisfied with a protection measure. 2.5 Social Media Analytics in Opinion Mining and Sentiment Analysis Social media (SM) analytics combines, extends, and adapts methods for the analysis of social media data (Stieglitz et al., 2014; Zeng, et al., 2010, Stieglitz et al., 2018) and involves the following three steps: 1) capturing relevant social media data by monitoring or listening, to Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 6 various social media sources, archiving relevant data and extracting pertinent information; 2) understanding the data (after removing noise) by applying a key technique, such as sentiment analysis or social network analysis and finally, 3) presenting findings in a meaningful way (Fan & Gordon, 2014). The area encompasses a variety of modelling and analytical techniques such as topic modelling and opinion mining (Fan & Gordon, 2014). Opinion mining or sentiment analysis is the “computational study of opinions, feelings and subjectivity in text” (Pang & Lee, 2008). It leverages computational linguistics, natural language processing, and other text analytics methods (Fan and Gordon, 2014). Social media platforms have enabled users to easily express and share their thoughts and opinions. Thus, SM platforms constitute a valuable resource for a variety of applications requiring insights on the public opinion about a concept. In recent years, sentiment analysis in the micro- blogging site Twitter has attracted a lot of attention from researchers as many users share opinions, thoughts, and, in general, any kind of information about any topic of their interest on this platform (Giachanou & Crestani, 2016). Opinions expressed in tweets can be useful for enterprises, governments as well as users of products and services (Giachanou & Crestani, 2016). For instance, based on the understanding of public views, expressed through tweets, regarding different social issues a government can initiate prompt actions. Similarly, potential customers of a product can decide whether to buy a product or not based on opinionated information. Emotion detection or identifying various emotions from text, is a problem related to Twitter sentiment analysis. While sentiment reflects a feeling, emotion reflects an attitude (Tsytsarau & Palpanas, 2012). Twitter posts have been analysed to understand the interplay between macroscopic socio-cultural events, such as the outcome of a political election, and the public’s mood state (Bollen et al., 2011). Much of the work on emotion analysis focuses on the six Ekman emotions of joy, sadness, anger, fear, disgust, and surprise (Ekman, 1992) whereas some others also focus on complex emotions such as politeness, rudeness, embarrassment, deception, confidence and confusion (Mohammad, 2012). In this work, we analyse privacy-related positive and negative emotions around contact tracing apps to understand user perceptions on privacy risks and protection for such apps. 2.6 Gaps in extant literature and the contributions of this research Till date, researchers have captured privacy attitudes and concerns of users through conventional survey-based approaches using questionnaires (Malhotra et al., 2004; Balapour et al., 2020; Xu et al., 2005; Jarvenpaa et al., 2000; Buchanan et al., 2007). Survey-based methods in privacy have several limitations, including limited sample size, availability mostly in English, and costly to conduct in a multinational or global format (Ur & Wang, 2013; Gonzalez et al., 2019b). Thus, recent works (Raber & Kruger, 2018; Gonzalez et al. 2019a, 2019b; Fiesler & Hallinan, 2018) have begun to explore text mining as an alternative approach. Social media (SM) platforms are popular spaces for users to share their candid opinions and experiences and have large amounts of cross-cultural user-generated content (UGC) (Rathore & Ilavarsan, 2020; Gonzalez et al, 2019b). As opposed to asking users directly about their privacy concerns as a part of a research study, this source of data, being a participant-driven response, can provide authentic and accurate reactions to real situations revealing issues that matter to the users (Fiesler & Hallinan, 2018). SM has the potential to capture users’ behavioural intentions by obtaining emotions (Rathore & Ilavarsan, 2020) related to various issues such as the introduction of contact tracing apps by the government. Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 7 Since emotions are short-term responses, compared to other traditional approaches (such as surveys and reviews in newspapers) to assess such responses, SM platforms can gauge emotions faster (Rathore & Ilavarsan, 2020). In addition, compared to traditional methods, SM provides information at a lower cost, in real-time and UGC is sufficient in terms of the amount of data, the relative lack of bias, the cost of the data and enables the unique opportunity to analyse cross-cultural, multi-language data (Rathore and Ilavarsan, 2020; Gonzalez et al. 2019b). Reference Examination of user’s perceived privacy risks/ concerns Examination of user’s perceived privacy protections Comparison across geographies Examination of UGC on social media Analysis of users’ privacy- related emotions Context of digital contact tracing Our work       Fox et al. (2021)  × × × ×  Chan & Saqib (2021)  ×  × ×  Lin et al. (2021)  × × × ×  Walrave et al. (2020)  × × × ×  Horvath et al. (2020)   × × ×  Praveen et al., (2020a) × ×   ×  Crable & Sena (2020) × × ×  ×  Praveen et al. 2021 × ×   ×  Simko et al. (2020)  × × × ×  Gonzalez et al. (2019a)  ×   × × Gonzalez et al. (2019b)  ×   × × Table 1. Research gaps and our contributions Several recent research works have highlighted numerous privacy concerns on contact tracing apps such as profiling and tracking of specific users (Wen et al., 2020; Bengio et al., 2021; Leith & Farrell, 2020), de-anonymization of infected persons (Baumgärtner et al., 2020), data misuse by third parties (Azad et al., 2020) and government surveillance (Rowe, 2020; Fahey & Hino, 2020). Some studies have explored how privacy concerns have affected the willingness of citizens to adopt contact tracing apps in different countries such as Ireland, France, Australia and Belgium. Within this literature, most works have adopted a survey- based approach (Fox et al., 2021; Chan & Saqib, 202; Walrave et al., 2020; Horvath et al., 2020). Citizens’ privacy concerns have been demonstrated to have a negative impact on their willingness to adopt contact tracing apps. A limited number of studies have also been conducted to understand citizen opinion on contact tracing in different countries. While in Ireland, citizens exhibited mostly positive opinion about contact tracing (Rekanar et al., 2020), in some other countries such as India and Brazil, citizens exhibited more negative opinions (Praveen et al., 2020a; Praveen et al., 2021). Privacy concerns have emerged as one of the major reasons behind negative feelings towards contact tracing (Rekanar et al., 2020; Praveen et al., 2020a; Praveen et al., 2021; Crable & Sena, 2020) although such concerns have Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 8 not been explored further by these works. Our work builds on the existing literature but is different in the following ways. First, we show how emotional information embedded in user-generated content (UGC) from social media platform can be elicited to gain privacy- related insights for a given technology or platform. Specifically, through emotion analysis of tweets, we attempt to understand citizen privacy perceptions about the Indian contact tracing app Aarogya Setu and the Singaporean contact tracing app TraceTogether which were among the first apps of this kind to be launched. Secondly, we examine users’ privacy- related emotions on perceived privacy risks and perceived privacy protections surrounding these apps thus assessing the negative beliefs (privacy concerns) of users in the privacy calculus model. In the broader context, this privacy calculus would directly affect the willingness to disclose personal data and hence the acceptance of contact tracing apps. Thirdly, we compare the results for apps introduced by two different countries to understand how these emotions varied for these apps. To the best of our knowledge, we are the first to examine privacy-related emotions on contact tracing apps. We also extend the research on how UGC on SM can be mined using text-Convolutional Neural Network (text-CNN) to derive privacy insights on contact tracing apps. We have organized and presented our contributions vis-à-vis key related research works in Table 1. 3 Method and Data We develop a method enabling fine-grained extraction of user’s positive and negative emotions around possible privacy harms from contact tracing apps from UGC to understand their perceived privacy risks and perceived privacy protections without relying on traditional, survey-based methods. Perceived privacy risk is represented by a negative emotion around a privacy harm related topic concerning the app. Similarly, perceived protection is represented by a positive or a negative emotion around a privacy protection topic concerning the app. We adopt social media analytics for Twitter data collection and analysis. SM analytics provides many cost-effective and real-time approaches to gain an understanding of user sentiments (Rathore & Ilavarsan, 2020; Rathore et al., 2021). In Twitter, UGC is mainly textual content in the form of short posts, called tweets. The character restriction on tweets makes them highly expressive compared to other social media posts (Pak & Paroubek, 2010; Singh et al., 2019). In the remainder of this section, we present the approach we adopted to extract privacy related insights from Twitter and specific methods used to analyse and visualise data. We develop a novel text-Convolutional Neural Network (text-CNN) to perform emotion analysis using transfer learning on the collected dataset. A collocation analysis is then performed to retrieve relevant bigrams and trigrams and calculate their association measures using PMI and chi-square test. User emotions towards the app are identified through tweets relevant to unigrams, bigrams, and trigrams that fall in the vicinity of privacy related terms. 3.1 Creation of Target Datasets Data pre-processing begins with the selection of the event. In our case, it is the launch of contact tracing apps, Aarogya Setu in India, and TraceTogether in Singapore. Enough data was available and could be extracted for our choice of event for further analysis. Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 9 Figure 1. A tweet on Aarogya Setu Figure 2. A tweet on TraceTogether In the first stage of data pre-processing, suitable keywords were selected. This was followed by the second stage where tweets were extracted, and the third stage consisted of tweet preparation. The data source for UGC is the micro-blogging site Twitter. The keywords selected for extracting tweets related to the Aarogya Setu app include Aarogya Setu, aarogyasetu, and various other combinations of these terms. Similarly, keywords were chosen for extracting tweets related to the TraceTogether app. We extracted historical tweets from the Twitter API. A list of individual tweet IDs were first obtained and then hydrated using a custom hydrating algorithm. All the extraction and hydration processes were carried out using Python 3 on Google Colab. All tweets that include the app’s name in their text, mention, or hashtag were collected for the period beginning with the launch of that app. For Aarogya Setu, tweets between the dates 2nd April, 2020 to 28th February, 2021 and for TraceTogether, tweets between the dates 29th March, 2020 to 28th February, 2021 were collected. Sample tweets for Aarogya Setu and TraceTogether have been shown in Figures 1 and 2 respectively. Twitter data contains many languages, unstructured abbreviations, and grammar. Therefore, after retrieving tweets, it is necessary to convert them into a meaningful format. Firstly, non- ASCII characters were removed, followed by eliminating punctuations, numeric values, and URLs. Unnecessary words referred to as stop words (prepositions, determiners, conjunctions, etc.) that do not contribute to our analysis were filtered out using Natural Language Toolkit (NLTK). Next, the entire corpus was lemmatized to keep only root words for analysis. We only considered tweets in English. Ultimately, our first dataset consisted of 89005 tweets related to Aarogya Setu and the second dataset contained 10997 tweets related to TraceTogether. These target datasets were annotated to evaluate the performance of our approach. Each item is labelled into one of the nine emotion categories: anger, disgust, fear, guilt, sadness, shame, surprise, happiness, and neutral. The datasets were strongly labelled by six human annotators. The annotators were selected based on their domain knowledge, linguistic knowledge, and prior annotation experience. Considering the huge size of the target datasets, the annotation task was divided into two sets of 50K tweets each. Each set was labelled by three individual annotators. The intercoder reliability scores, calculated to measure the agreement among the annotators, accounted for 91% and 96% for Aarogya Setu and TraceTogether, respectively. Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 10 3.2 Creation of Auxiliary Dataset We prepare an auxiliary sample set for training the proposed algorithm by combining publicly available datasets pertaining to the emotion analysis task. We combine eight publicly available text-emotion analysis datasets to form a single large auxiliary set. The datasets used in the combination are ISEAR1, DailyDialog2, EmotionStimulus3, SemEval 20194, EmoBank5, EmoInt6, CrowdFlower7, and Emotion Lines8. Table 2 depicts the size and labels in each of these eight datasets. Dataset Size Labels ISEAR 7473 sentences Joy (1081), sadness (1067), fear (1081), anger (1071), guilt (1052), disgust (1067) and shame (1054) DailyDialog 102,879 Dialogues extracted from conversations Happiness (12885), sadness (1150), anger (1022), disgust (353), fear (74), surprise (1823), and others (85572) EmotionStimulus 2414 sentences Anger (483), disgust (95), fear (423), happy (479), sad (575), shame (146), surprise (213) SemEval 2019 38,424 Happy (4669), Sad (5838), Angry (5954), Others (21,963) EmoBank 10k news headline, essays, blogs, newspapers, fiction, letters and travel guides Dimensional labelling according to Valence (V), Arousal (A) and Dominance (D) with numeric values in the range [1,5] Relabeled as: Anger (516), Disgust (62), Happy (3584), Neutral (2629), Sad (379), Shame (119), Surprise (2773) EmoInt 3960 items Anger (941), Fear (1257), Joy (902), Sadness (860) CrowdFlower 39,998 tweets Anger (110), Boredom (179), Empty (827), Enthusiasm (759), Fun (1776), Happiness (5209), Hate (1323), Love (3842), Neutral (8638), Relief (1526), Sadness (5165), Surprise (2187), Worry (8459) Emotion Lines (EL) 29,245 utterances obtained from dialogue in Friends TV Show and Facebook messenger chats. Friends: Neutral (6531), Joy (1710), Sad (497), Fear (247), Anger (759), Surprise (1658), Disgust (331), Non-neutral (2772) EmotionPush: Neutral (9855), Joy (2101), Sad (514), Fear (40), Anger (140), Surprise (568), Disgust (106), Non- neutral (1418) Table 2. Details of datasets used in auxiliary set To ensure a definite scale of emotions, the auxiliary dataset is filtered to contain a specific set of nine emotion labels: happiness, sadness, neutral, fear, anger, disgust, guilt, shame, and surprise. These nine labels broadly cover the type of emotions perceived by a human being. However, the datasets constituting the auxiliary set contain varied labels. We map these items with multiple emotion labels to the desired nine label categories, as described in Table 1 https://www.kaggle.com/shrivastava/isears-dataset (accessed on 20-Feb-2021) 2 https://www.aclweb.org/anthology/I17-1099/ (accessed on 20-Feb-2021) 3 https://www.site.uottawa.ca/~diana/resources/emotion_stimulus_data/ (accessed on 20-Feb-2021) 4 https://www.humanizing-ai.com/emocontext.html (accessed on 15-Feb-2021) 5 https://github.com/JULIELab/EmoBank (accessed on 10-Feb-2021) 6 http://saifmohammad.com/WebPages/EmotionIntensity-SharedTask.html (accessed on 10-Feb-2021) 7 https://www.crowdflower.com/wp-content/uploads/2016/07/text_emotion.csv (accessed on 10-Feb-2021) 8 https://sites.google.com/view/emotionx2019/datasets (accessed on 10-Feb-2021) https://www.aclweb.org/anthology/I17-1099/ https://github.com/JULIELab/EmoBank http://saifmohammad.com/WebPages/EmotionIntensity-SharedTask.html Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 11 3. Items labelled as joy in ISEAR, EmoInt, and Emotion Lines are categorized as happiness in the auxiliary set. The remaining items labelled with emotions – anger, disgust, fear, guilt, sadness, shame, surprise, and neutral are used with the same labels in the auxiliary set. The EmoBank dataset originally contains dimensional labelling by providing a score between 1 to 5 for three attributes – Valence (V), Arousal (A), and Dominance (D). Its mapping on the emotional scale was developed by the annotators categorizing the items into anger, disgust, sadness, shame, surprise, happy, and neutral. The categorization of an item in each emotional category is carried out according to the rules described in Table 3. The items labelled with emotions other than the nine categories that we use, such as boredom, empty, enthusiasm, fun, hate, love, relief, worry, others, and non-neutral were not used in the auxiliary set. Datasets Anger Dis- gust Fear Guilt Sadness Shame Surprise Happiness Neutral ISEAR Anger Dis- gust Fear Guilt Sadness Shame - Joy - DailyDialog Anger Dis- gust Fear - Sadness - Surprise Happiness - Emotion- Stimulus Anger Dis- gust Fear - Sad Shame Surprise Happy - SemEval 2019 Angry - - - Sad - - Happy - EmoBank 2.5≥V, D<3, A≥3 2.5≥V, D≥3, A<3 - - 2.5≥V, D≥3, A≥3 2.5≥V, D<3, A<3 {V≥3, D≥3, A≤3} + {V≥3, D<3} {V≥3, D≥3, A≥3} {2.5> V>3} EmoInt Anger - Fear - Sadness - - Joy - CrowdFlower Anger - - - Sadness - Surprise Happiness Neutral EL (Friends) Anger Disgust Fear - Sad - Surprise Joy Neutral EL (EmotionPush) Anger Disgust Fear - Sad - Surprise Joy Neutral Table 3. Emotion Mapping Emotion Category No. of items in Auxiliary Dataset No. of items in Target Dataset- AarogyaSetu No. of items classified by text-CNN in AarogyaSetu No. of items in Target Dataset- TraceTogether No. of items classified by text-CNN in TraceTogether Anger 10996 20894 21540 1699 1734 Disgust 2014 1373 890 356 448 Fear 3122 21886 22251 1642 1786 Guilt 1052 325 89 138 22 Sadness 16045 3221 2850 312 169 Shame 1319 336 180 81 34 Surprise 9222 4673 3735 300 277 Happiness 32620 24656 25010 4940 5021 Neutral 27653 11641 12460 1529 1506 Table 4. Number of items in each emotion category upon annotation and classification Table 4 depicts the number of text items in the auxiliary dataset and the target datasets for each emotion category. It also illustrates the number of items in each emotion class according to manual annotation and text-CNN classification. It demonstrates that the auxiliary dataset is rich in items with ‘Anger’, ‘Sadness’. ‘Happiness’, and ‘Neutral’ labels. However, the remaining emotions on a fine-grained scale contain lesser instances. This data imbalance is Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 12 due to low availability of annotated emotion classification datasets. However, the auxiliary dataset is sufficient to train a deep neural network effectively. 3.3 Proposed Method In this section, we discuss the method we adopted to derive privacy insights from user tweets. It consists of collocation analysis and emotion analysis. The method was applied separately for Aarogya Setu and TraceTogether datasets. Collocation analysis is performed to analyse linguistic, syntactic, and lexical relations in sentences to identify words that occur together more often than expected. We first analysed the meaningful bigrams (two words taken together) and trigrams (three words taken together) extracted from the collected dataset. Collocations are retrieved using NLTK’s bigram and trigram finders. The method returns all significant co-occurrences of terms with their frequencies or association score. For computing association score, Pointwise Mutual Information (PMI) is calculated. Mutual information is an information theoretic concept (Church & Hanks, 1990). If two points or words, x and y have probabilities P(x) and P(y), then their PMI, I (x, y) is defined as 𝐼𝐼(𝑥𝑥,𝑦𝑦) = 𝑙𝑙𝑙𝑙𝑙𝑙2 𝑃𝑃(𝑥𝑥,𝑦𝑦) 𝑃𝑃(𝑥𝑥)𝑃𝑃(𝑦𝑦) Thus, PMI compares the probability of observing x and y together with the probabilities of observing x and y independently. If there is an association between x and y, then the joint probability P(x, y) will be much larger than the chance probability P(x)P(y) leading to I(x,y)≫0. On the other hand, if there is no such association between x and y, then P(x, y) ≈ P(x)P(y) and thus I(x, y) ≈ 0. Chi-square test is used as another measure to calculate the association among bigrams and trigrams. Using collocation analysis, we obtain bigrams and trigrams along with their frequency of usage, PMI score and chi-square test of co-occurrence. A positive PMI value signifies that the number of co-occurrences of the individual words as bigrams and trigrams is slightly less than the number of occurrences of those words individually or greater than expected co- occurrences. A negative PMI value means that the words in bigrams and trigrams occur together significantly less than they occur individually. We use the chi-sq test as a criterion for our analysis where chi-sq test statistic value is greater than 3.841 for bigrams and greater than 5.991 for trigrams holds the significance of their co-occurrence. The bigrams and trigrams with higher frequency, greater PMI score, and a higher chi-sq statistic value represent what users discussed the most about in the tweets. We employed transfer learning to perform emotion classification on our datasets. Transfer learning is beneficial when it is required to transfer knowledge-based features to a target domain. This method is helpful in circumstances where data is not annotated, and transfer learning can be used to perform unsupervised classification on the target dataset by learning the feature representations from a similar auxiliary labelled dataset. The proposed method offers an advantage over traditional training and testing procedure of a machine learning algorithm on a labelled dataset. In supervised classification scenarios, data requires high quality manual annotations which is costly and resource expensive in the real-world scenario. Also, in situations such as the recent pandemic, where credible research results are critically needed, it is not feasible to readily collect, annotate, and devise algorithms for early classification. The proposed transfer-learning-based method is advantageous given that the Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 13 algorithm uses transferred domain knowledge obtained upon training a neural network on a large-sized auxiliary dataset which can be readily used to generate predictions for any unlabelled real-time testing data upon performing fine-grained emotion classification. Figure 3. Architectural representation of proposed Text-CNN for emotion analysis The proposed architecture of text-based convolutional neural network (CNN) for emotion analysis is demonstrated in Figure 3. Initially, the proposed Text-CNN classifier is trained on a large set of labelled samples, the auxiliary dataset. Then, textual feature representations obtained from training on the auxiliary set are transferred to the target domain to label the target set. The auxiliary dataset is split in the ratio 8:1:1 for training, validation, and testing, respectively. The proposed convolutional neural network is trained on the auxiliary dataset for 20 epochs with a batch size of 64. The weighted instances from the auxiliary set are used to extract labels on the target dataset of tweets. The entire framework was built and implemented on Google Colab using Python 3 with an allocated RAM of 13.53 GB and NVIDIA Tesla K80 GPU. 3.4 Experimental Performance We discuss the experimental performance of our transfer learning-based text-CNN approach to establish the efficacy of the proposed algorithm. The results are demonstrated for both AarogyaSetu and TraceTogether datasets through confusion matrices, and four evaluation metrics - accuracy, precision, recall, and f1-score. A detailed error analysis is also performed to identify the possible causes of misclassification of tweets. Tables 5 and 6 represent the confusion matrices for AarogyaSetu and TraceTogether datasets, respectively. Table 7 displays the class-wise precision, recall and f1-scores for emotion classification for AarogyaSetu and TraceTogether datasets. It is evident from figure 5 that ~80% of the tweets from each category are correctly classified for AarogyaSetu. For the TraceTogether dataset, happiness and neutrality is correctly identified for most of the tweets, however guilt and shame are poorly classified due to the low number of tweets in these emotion categories. The testing accuracies on AarogyaSetu and TraceTogether are found to be 83.022% and 89.406%, respectively. This performance is exceptionally well given that the classification model uses domain transferrable knowledge for training instead of traditional supervised learning. The accuracy scores demonstrate the proposed method’s efficacy to R Input Text Happiness Sadness Fear Anger Disgust Guilt Shame Surprise D ro po ut So ftm ax Embedding Layer Convolution + ReLU layers Max Pooling Dense Re LU A ct iv at io n K ar na ta ka h ig h co ur t b ar s C en tre fr om sh ar in g A ar og ya Se tu ap p da ta w ith ou t u se r c on se nt Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 14 generate fine-grained emotion labels on unseen real-world data. However, accuracy is not an appropriate measure when the evaluation dataset is imbalanced. To have a better picture of the model’s efficiency, we calculate the f1-scores for classification on both datasets. From table 5, we can analyse the class-wise performance observing that all the categories achieve f1-scores in the range of 75% to 87% for AarogyaSetu. Similarly, anger, fear, sadness, surprise, happiness, and neutrality fall in the f1-score range of 61% to 91% for TraceTogether. Low f1- scores are obtained for the disgust, guilt, and shame categories with 57.69%, 26.96%, and 40.57%, respectively. Anger Disgust Fear Guilt Sad Shame Surprise Happy Neutral Anger 0.82 0.02 0.06 0.00 0.01 0.00 0.01 0.04 0.03 Disgust 0.06 0.79 0.01 0.02 0.02 0.02 0.03 0.00 0.06 Fear 0.05 0.01 0.83 0.00 0.01 0.00 0.02 0.06 0.02 Guilt 0.02 0.04 0.02 0.80 0.04 0.02 0.02 0.00 0.02 Sad 0.04 0.01 0.03 0.00 0.79 0.01 0.01 0.08 0.02 Shame 0.06 0.04 0.00 0.02 0.01 0.82 0.00 0.02 0.02 Surprise 0.04 0.01 0.04 0.00 0.01 0.00 0.84 0.04 0.02 Happy 0.04 0.00 0.05 0.00 0.01 0.00 0.03 0.85 0.01 Neutral 0.07 0.00 0.05 0.00 0.00 0.00 0.00 0.07 0.80 Table 5. Confusion matrix for AarogyaSetu dataset Anger Disgust Fear Guilt Sad Shame Surprise Happy Neutral Anger 0.86 0.03 0.02 0.01 0.04 0.01 0.01 0.00 0.01 Disgust 0.16 0.53 0.05 0.03 0.07 0.03 0.02 0.00 0.11 Fear 0.05 0.01 0.86 0.02 0.02 0.00 0.01 0.01 0.02 Guilt 0.09 0.05 0.14 0.23 0.27 0.14 0.00 0.00 0.09 Sad 0.07 0.04 0.02 0.05 0.66 0.07 0.02 0.02 0.05 Shame 0.03 0.12 0.06 0.32 0.06 0.32 0.03 0.00 0.06 Surprise 0.07 0.05 0.08 0.01 0.00 0.01 0.71 0.06 0.00 Happy 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.97 0.01 Neutral 0.00 0.01 0.01 0.01 0.03 0.01 0.00 0.02 0.91 Table 6. Confusion matrix for TraceTogether dataset Class Aarogya Setu Results (%) Trace Together Results (%) Precision Recall F1-Score Precision Recall F1-Score Anger 82.27 84.81 75.24 86.10 87.88 73.65 Disgust 78.65 50.98 81.94 52.68 66.29 57.69 Fear 83.07 84.46 79.33 86.39 93.97 77.27 Guilt 79.78 21.85 86.18 22.73 3.62 26.96 Sad 79.19 70.07 82.48 65.68 35.58 61.12 Shame 82.22 44.05 86.99 32.45 13.58 40.57 Surprise 84.34 67.41 85.72 70.76 65.33 78.70 Happy 85.39 86.61 78.86 96.91 98.50 92.81 Neutral 80.33 85.98 79.93 91.04 89.67 80.15 Table 7. Results of emotion classification on AarogyaSetu and TraceTogether datasets To analyse the potential causes of misclassification, we demonstrate the error analysis for AarogyaSetu and TraceTogether in Table 8 and Table 9, respectively. We calculate the total number of items misclassified for each category and list the count of misclassified tweets in Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 15 each of them. Class-wise error percentage is also calculated and listed in the tables. We also try to provide possible causes or hypothesis for some of the misclassifications. For AarogyaSetu, a total of 3820 anger tweets out of 20,894 were misclassified with the maximum misclassification of 1331 tweets (34.84% of total errors in anger classification) in the Fear category. It was followed by 807 tweets (21.13%) misclassified as happiness. Taking these two numbers as matter of concern, we hypothesize that some parts of speech might deceive the algorithm to misclassify anger as fear, but some strongly used words in phrases like “Joy turned downhill” were causes for misclassification of Anger as Happiness. Other misclassification account to 16.94% as Neutral, 9.4% as disgust, 8.25% as sadness, 1.36% as guilt, and 0.65% as shame. The surprise emotion can be both positive as well as negative accounting to 7.43% of the anger misclassification. Out of 1373 disgust tweets, 190 misclassifications are present. 50 tweets each have been misclassified as Anger and Neutral. Sadness and Shame labels take 21 tweets each as misclassifications, whereas 23 tweets are misclassified as surprise followed by 15 tweets being misclassified as guilt, 8 as fear and 2 as happiness. For the Fear emotion, the algorithm misclassified 32.94% of tweets as Happiness followed by 28.30% as Anger. Similar reasons were observed for Fear being misclassified as Anger as was for the vice-versa. The algorithm cannot distinguish emotion mixed with sarcasm and thus tweets like “Secularism is like AarogyaSetu app. You have to declare yourself that you are sick or well irrespective of what actually you are” these are mapped as happiness instead of appropriate emotion, which in this case would be ‘Fear’. Other misclassifications were relatively insignificant in proportion and could not be attributed with a specific cause. The smaller quantity of tweets related to Guilt and Shame emotions makes them both difficult and unnecessary for error analysis. 39.46% of the tweets with Sadness emotion were misclassified as Happiness. This can be attributed to tweets that contained multiple positive words against lesser number of negative words despite the context of the tweet conveying a sad emotion. One such example is the tweet “I agree with you fully. In India, funding for R & D is driven by success, profit, negative mindset to serve vested interests. In West, it is driven by vision, mission. The sound by negativists on the AarogyaSetu app is one such clear example.” This is followed by Anger with 16.86% and Fear with 14.84% as misclassification. Surprise can be both positive and negative and hence find similar proportion of misclassification as Fear (26.67%), Happiness (25.98%) and Anger (25.30%). Tweets with Happiness emotion are misclassified Majorly misclassified as Fear (32.89%), Anger (26.32%) and Surprise (20.27%). Higher proportions of negative words with positive context, was found to be the key reason for such misclassification. Some neutral tweets were randomly misclassified as Anger, Fear and Happiness in majority amongst others. For TraceTogether tweets, similar reasons can be attributed for the misclassifications. Tweets with Guilt, Sadness and Shame emotions were present in negligible proportions and are insignificant for error analysis. Tweets with Anger emotion were misclassified as Sadness (31.54% of total misclassifications), followed by Disgust (24.48%), and Fear (13.69%) with other negligible misclassifications. Tweets with Disgust emotion were majorly misclassified as Anger (33.96%), Neutral (24.06%) followed by sadness (14.62). Fear emotion tweets were misclassified as Anger (33.33% of total misclassification in the Fear category), Guilt (18.11%), and Neutral (15.23%). Tweets with Surprise and Happiness emotions had higher accuracy and hence, less misclassification. Although, a bias can be observed where Tweets with Surprise emotion are Majorly misclassified as Fear (25.93%), Anger (24.69%) and Happiness Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 16 (22.22%) like the AarogyaSetu tweets, which shows the positive and negative variation in surprise emotion. Tweets with happiness emotion were majorly misclassified into Surprise (41.94%). Anger classified as Disgust Fear Guilt Sadness Shame Surprise Happiness Neutral Count 359 1331 52 315 25 284 807 647 % of total Errors 9.40 34.84 1.36 8.25 0.65 7.43 21.13 16.94 Disgust classified as Anger Fear Guilt Sadness Shame Surprise Happiness Neutral Count 50 8 15 21 21 23 2 50 % of total Errors 26.32 4.21 7.89 11.05 11.05 12.11 1.05 26.32 Fear classified as Anger Disgust Guilt Sadness Shame Surprise Happiness Neutral Count 1066 148 102 330 46 390 1241 444 % of total Errors 28.30 3.93 2.71 8.76 1.22 10.35 32.94 11.79 Guilt classified as Anger Disgust Fear Sadness Shame Surprise Happiness Neutral Count 2 4 2 4 2 2 0 2 % of total Errors 11.11 22.22 11.11 22.22 11.11 11.11 0.00 11.11 Sadness classified as Anger Disgust Fear Guilt Shame Surprise Happiness Neutral Count 100 27 88 10 42 29 234 63 % of total Errors 16.86 4.55 14.84 1.69 7.08 4.89 39.46 10.62 Shame classified as Anger Disgust Fear Guilt Sadness Surprise Happiness Neutral Count 10 8 0 4 2 0 4 4 % of total Errors 31.25 25.00 0.00 12.50 6.25 0.00 12.50 12.50 Surprise classified as Anger Disgust Fear Guilt Sadness Shame Happiness Neutral Count 148 27 156 8 23 4 152 67 % of total Errors 25.30 4.62 26.67 1.37 3.93 0.68 25.98 11.45 Happiness classified as Anger Disgust Fear Guilt Sadness Shame Surprise Neutral Count 962 65 1202 63 229 38 741 355 % of total Errors 26.32 1.78 32.89 1.72 6.27 1.04 20.27 9.71 Neutral classified as Anger Disgust Fear Guilt Sadness Shame Surprise Happiness Count 836 35 615 0 40 10 54 861 % of total Errors 34.11 1.43 25.09 0.00 1.63 0.41 2.20 35.13 Table 8. Error analysis for emotion classification on AarogyaSetu data Anger classified as Disgust Fear Guilt Sadness Shame Surprise Happiness Neutral Count 59 33 21 76 14 11 4 23 % of total Errors 24.48 13.69 8.71 31.54 5.81 4.56 1.66 9.54 Disgust classified as Anger Fear Guilt Sadness Shame Surprise Happiness Neutral Count 72 22 15 31 12 7 2 51 % of total Errors 33.96 10.38 7.08 14.62 5.66 3.30 0.94 24.06 Fear classified as Anger Disgust Guilt Sadness Shame Surprise Happiness Neutral Count 81 19 44 28 8 14 12 37 % of total Errors 33.33 7.82 18.11 11.52 3.29 5.76 4.94 15.23 Guilt classified as Anger Disgust Fear Sadness Shame Surprise Happiness Neutral Count 2 1 3 6 3 0 0 2 % of total Errors 11.76 5.88 17.65 35.29 17.65 0.00 0.00 11.76 Sadness classified as Anger Disgust Fear Guilt Shame Surprise Happiness Neutral Count 12 7 4 9 11 3 3 9 % of total Errors 20.69 12.07 6.90 15.52 18.97 5.17 5.17 15.52 Shame classified as Anger Disgust Fear Guilt Sadness Surprise Happiness Neutral Count 1 4 2 11 2 1 0 2 % of total Errors 4.35 17.39 8.70 47.83 8.70 4.35 0.00 8.70 Surprise classified as Anger Disgust Fear Guilt Sadness Shame Happiness Neutral Count 20 13 21 4 1 3 18 1 Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 17 % of total Errors 24.69 16.05 25.93 4.94 1.23 3.70 22.22 1.23 Happiness classified as Anger Disgust Fear Guilt Sadness Shame Surprise Neutral Count 11 7 5 16 11 7 65 33 % of total Errors 7.10 4.52 3.23 10.32 7.10 4.52 41.94 21.29 Neutral classified as Anger Disgust Fear Guilt Sadness Shame Surprise Happiness Count 7 10 9 13 46 12 3 35 % of total Errors 5.19 7.41 6.67 9.63 34.07 8.89 2.22 25.93 Table 9. Error analysis for emotion classification on TraceTogether data 4 Results and Discussion In this section, we present the results of our analysis of AarogyaSetu and TraceTogether datasets. To understand users’ opinions from tweets related to AarogyaSetu and TraceTogether app, we examined what they talked about the most and how they felt about it. For the first part, we used NLP-based collocation analysis, and for the latter part, we used deep learning-based emotion analysis. The four emotions - anger, fear, happy and neutral - account for 91.3% and 91.4% of tweets for AarogyaSetu and TraceTogether apps, respectively (see Figure 4). The emotions of anger and fear together account for 49.2% of the AarogyaSetu tweets whereas the happy emotion accounts for just 28.1% (Figure 4). For TraceTogether, the emotions of anger and fear together account for 32% of the tweets, whereas the happy emotion accounts for 45.7% of the tweets (Figure 4). So, overall, TraceTogether tweets show more happy emotions compared to AarogyaSetu tweets. Figure 4. Emotion classification of AarogyaSetu and TraceTogether tweets Tweets with neutral emotions hold negligible significance in the analysis, and therefore we end up with 76545 AarogyaSetu tweets and 9491 TraceTogether tweets for analysis. To 21540 1734 890 448 22251 1786 89 22 2850 169180 34 3735 277 25010 5021 12460 1506 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% AarogyaSetu TraceTogether Pe rc en ta ge o f t w ee ts Emotion Classification of Tweets Anger Disgust Fear Guilt Sad Shame Surprise Happy Neutral Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 18 simplify the study, we form two groups of reactions: positive and negative. Tweets with anger and fear emotion belong to the negative reaction group, while tweets with happy emotion belong to the positive reaction group. The remaining five (5) emotions — disgust, guilt, sad, shame and surprise, constitute a tiny proportion of tweets and were excluded to avoid discrepancies in the analysis Bigrams like ‘aarogya,’ ‘setu,’ and ‘use,’ ‘tracetogether’ appear more frequently than others because they resemble or contain the app's name. Trigrams like ‘ravi,’ ‘shankar,’ ‘prasad,’ also reflected in the bigram as ‘ravi,’ ‘shankar,’ which refers to a person’s name used in AarogyaSetu tweets have higher co-occurrences than their occurrence (frequency of ‘ravi,’ ‘shankar,’ ‘prasad’ is 1). In other words, such bigrams and trigrams are used more frequently together than used individually or used only together and therefore appear with higher PMI and Chi-sq statistic value. Table 10. PMI score and Chi-sq test statistic value of relevant bigrams and trigrams (AarogyaSetu) To eliminate this clutter of bigrams and trigrams from the analysis, we define sixteen words (keywords) to obtain insights into perceived privacy risks and perceived privacy protections of the users. These keywords are privacy, security, tracing, surveillance, risk, data, protection, steal, leak, sharing, policy, regulation, location, hacker, breach, and cybersecurity. Fourteen bigrams and thirteen trigrams for tweets of AarogyaSetu and five bigrams and five trigrams for the TraceTogether app were selected based on these keywords. Identifying the emotions of the tweets containing these keywords, bigrams, and trigrams would provide insight into public opinion regarding these contact tracing apps. 47.94% (36700 out of 76545) S.No Category Word / Bigram / Trigram PMI Score Chi-sq Statistic 1 Bigrams data protection 8.03 45543.46 2 data security 8.89 49722.38 3 data leak 8.52 46631.13 4 personal data 7.28 38550.22 5 privacy concern 8.46 46115.6 6 breach aarogya 6.91 21242.49 7 security breach 7.91 46805.46 8 contact trace 8.51 104729.3 9 cybersecurity cyberthreat 10.02 72836.47 10 personal information 9.33 28317.33 11 data breach 7.13 33266.90 12 privacy breach 6.75 25378.84 13 tracking app 5.03 1314.2 14 ethical hacker 9.84 98650.59 15 contact trace app 16.52 3014099 16 architecture indulge surveillance 22.09 98372189 17 Trigrams claim cybersecurity activist 21.41 72891958 18 healthy person privacy 16.09 4064320 19 data breach aarogyasetu 11.64 129263.7 20 setu app useless 7.73 82488.55 21 hack aarogya app 7.52 260435.3 22 hacker claim privacy 16.06 860757.1 23 privacy first design 16.08 1385968 24 potential security issue 7.18 247359.5 25 hacker raise concern 16.06 248054.3 26 aarogya setu privacy 10.71 105678 27 assure ethical hacker 19.91 45394203 Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 19 of the AarogyaSetu unique tweets and 53.15% (5045 out of 9491) of the TraceTogether unique tweets contained one amongst these keywords, bigrams, and trigrams. Table 10 and Table 11 present the PMI score and Chi-sq test statistic value of relevant bigrams and trigrams for AarogyaSetu and TraceTogether respectively. We have used only those bigrams and trigrams that have a higher PMI score and Chi-sq statistic value indicating their higher and frequent usage in the tweets. Table 11. PMI score and Chi-sq test statistic value of relevant bigrams and trigrams (TraceTogether) 4.1 Emotion Analysis of AarogyaSetu Tweets Tweets containing keywords such as privacy, security, surveillance, risk, protection, steal, leak, hacker, cybersecurity, and breach expressed mostly with anger and fear (see Figure 5). Similarly, bigrams such as data breach, privacy breach, privacy concern, security breach and breach aarogya and trigrams such as architecture indulge surveillance, healthy person privacy, data breach aarogyasetu and aarogyasetu privacy expressed negative emotions (see Figure 6 and Figure 7). Table 12. AarogyaSetu tweets containing selected keywords and bigrams with high negative reaction S.No Category Word / Bigram / Trigram PMI Score Chi-sq Statistic 1 Bigrams privacy concern 7.48 11720.4 2 security issue 5.52 100.50 3 contact tracing 6.71 43337.04 4 data breach 3.31 196.57 5 law enforcement 11.11 79726.38 15 contact trace app 11.03 636702.8 16 government technology agency 18.45 7540231 17 Trigrams local law enforcement 21.82 81533074 18 criminal procedure code 19.20 21036631 19 for criminal investigation 16.02 9596831 Keyword/ Bigram Tweet privacy It isn’t just about privacy with Aarogya Setu. The massive amounts of data as with any citizen info set, is prime target for profiling, cyber crime and even espionage. And the threats can come from some of the most formidable of threat actors. steal AarogyaSetu is a private sector effort built by the private for the private and of private resources. Nandan Nilekani calls this stealing of your personal data, a data democracy. policy AarogyaSetu is developed on a Privacy-first policy. Your data is never shared with anyone. The app is our protective shield against Coronavirus during this critical time. Watch this video to know more! #IndiaFightsCorona @GoI_MeitY @rsprasad @NICMeity @PIB_India @MIB_India https://t.co/YwyqDsKiP4 data Karnataka high court bars Centre from sharing AarogyaSetu app data without user consent data breach ‘Who is accountable for data breach?’: Retd Justice Srikrishna to TNM on AarogyaSetu privacy concern Now that they banned Chinese apps under privacy concern. I believe that we should also consider adding “Aarogya Set” to that list. ethical hacker How can you support this AarogyaSetu farce despite ethical hackers saying in no uncertain terms that it is an intentional tool of mass surveillance! Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 20 Except for the bigram personal information, all other bigrams constitute negative reaction in the majority (>50%). Emotions of anger and fear contribute the highest in tweets that share opinions regarding security breach, are related to information shared by ethical hacker, data breach possibilities in AarogyaSetu app (breach aarogya) that lead to privacy breach, and on opinions related to contact tracing. On extending the keyword data to the bigram data breach, we observe an increase of fear emotion in tweets (see Figure 5 and Figure 6), also reflected by a sample tweet on this bigram (see Table 12) where a user has re-iterated the question raised by retired Justice Srikrishna on who is going to be accountable for a data breach. The tweets also show that claims made by ethical hackers (see sample tweet in Table 12) and news articles on privacy fuel negative emotions of anger and fear among citizens towards the app. Trigrams cover topics of contact tracing (contact trace app), architecture (architecture indulge surveillance) of the app, claims made by cybersecurity activists (claim cybersecurity activist) and ethical hackers (hacker claim privacy), about the privacy of individuals (healthy person privacy) who have set up their profile in the app due to mandatory reasons, and about data breach (data breach aarogyasetu). Tweets with these trigrams have a high percentage of negative reaction (see Figure 7). The higher PMI and Chi-sq statistic value of the trigrams make them topics of higher relevance (see Table 10), and correspondingly, the negative reaction in the tweets holds high significance. Significantly, the overall positive reaction is less than 20% even though the government claims the app to be of privacy first design via tweets. Tweets, related to privacy protection status of the app, mostly originating from news media or the government, and which contain keywords such as data, sharing, policy and location have most of the emotions labelled as happy (see sample tweets with data and policy in Table 12). Figure 5. Emotion distribution of keywords for AarogyaSetu and TraceTogether tweets Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 21 Overall, with respect to AarogyaSetu, users have expressed negative emotions not only related to possible privacy harms such as surveillance and leak of sensitive information but also towards the lack of accountability and suitable privacy protection measures. Overall, the negative emotion in these tweets either reflect user concerns towards potential privacy harms or the lack of protection measures and positive emotions towards protection measures considered sufficient by them. This re-iterates that user emotions can be seen as key informants of negative beliefs within the Privacy Calculus Theory. Ultimately, since more negative emotions were observed for AarogyaSetu compared to positive ones, users who have expressed their opinions via tweets may have a net negative belief towards the app which could eventually mean lower willingness to rely on the app. Figure 6. Emotion distribution of bigrams for AarogyaSetu and TraceTogether tweets Figure 7. Emotion distribution of trigrams for AarogyaSetu and TraceTogether tweets Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 22 4.2 Emotion Analysis of TraceTogether Tweets Tweets with the keywords surveillance, data, hacker, and breach expressed higher negative emotions compared to the other keywords (see Figure 5). Tweets with bigrams such as privacy concern, data breach and law enforcement and trigrams such as government technology agency, local law enforcement and criminal procedure code express higher negative emotions compared to other bigrams and trigrams respectively (see Figure 6 and Figure 7). Table 13. Sample tweets containing selected keywords and bigrams with high negative reaction (TraceTogether) Users were concerned about the app’s vulnerabilities even when the application code was made open source, mainly because these concerns were backed by ethical hackers who claimed that government authorities were still logging information without the user’s approval or awareness (see sample tweet on data in Table 13). They expressed fear and anger regarding their loss of privacy as claims have been made that data is accessible to the police ultimately reducing the app’s usage and reduction in trust on the government (see sample tweets on breach and data breach in Table 13). Tweets containing the keywords security, tracing, risk, protection, and location have a high positive reaction (see Figure 5). People have appreciated some of the privacy protection measures of the app including no collection and usage of location data, sensible information security features along with clear explanations (see sample tweets in Table 14). Overall, for TraceTogether, users have expressed positive emotions when they perceived a higher level of privacy protection and negative emotions when they perceived higher privacy risks as they came across information indicating that data is being shared without permission or that the protection measures are not sufficient. Once again, this re-iterates that user emotions can be seen as key informants of negative beliefs within the Privacy Calculus Theory. To summarize, higher positive emotions have been observed in TraceTogether tweets compared to AarogyaSetu tweets (Figure 4. The bigram contact tracing and trigram contact trace app have received more positive emotion in case of TraceTogether and more negative reaction in case of AarogyaSetu (see Figures 6 and 7). TraceTogether has also garnered more Keyword / Bigram Tweet sur- veillance The app cannot be examined easily using the conventional method. It is hard to verify the claims on privacy and raises concerns over #surveillance. Being open source doesn.t always mean the actual code of #TraceTogether either. #Singapore #COVID19 #coronavirus https://t.co/nA65xkoviT data coder posted analysis for TraceTogether on GitHub, saying he found pointers in the code to a government data-collection platform, raising the possibility that! authorities were logging the data without permission. https://t.co/51JrobHBr5 @WojciechKlicki @Apsalaar @bpaszcza breach #poll a huge debate going on whether the contact tracing app breaches privacy as it was claimed to be accessible by the police for crimes. since then, many have deleted the app. previously it was mandatory for most public places and facilities, including my office. #TraceTogether data breach On reading more responses, the general direction is that people will trust the SG govt LESS on any future issues. Was it worth having #TraceTogether data by breaching THE social contract? Which #sg govt body is looking at things at a systems level? Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 23 positive emotions than AarogyaSetu around keywords such as security, tracing, risk, protection and sharing (see Figure 5). Both the apps have received negative emotions around keywords such as surveillance, steal and hacker. Table 14. Sample tweets containing selected keywords with high positive reaction (TraceTogether) 5 Theoretical Contribution This study presents a methodological contribution wherein one can examine how user- generated content can be utilized to understand users’ perceived privacy risks and perceived privacy protections towards a potentially privacy-invasive technology. So long, researchers have adopted survey-based methods to understand user’s negative beliefs related to technology within the Privacy Calculus Theory. Compared to this traditional method, the approach described in the paper contributes to gathering spontaneous and candid opinions of users. Our approach can be extended to understand privacy perceptions of users for a variety of commercial as well as e-government applications. Unlike most of the literature related to user privacy perceptions which have focused on commercial technologies such as e- commerce and commercial mobile applications (Gu et al., 2017; Li, 2014), our emphasis is on the privacy concerns of top-down government health surveillance apps like contact tracing. Privacy Calculus Theory indicates that privacy-related decisions constitute rational anticipations of risks and benefits connected to data disclosure. We have incorporated privacy-related emotions as key informants of privacy concerns within the Privacy Calculus Theory. Our findings highlight users’ positive emotional response towards privacy protective measures and negative response towards potential privacy harms, further supporting studies which show that emotional states also impact privacy-related decision making. The insights from the findings for our second research question (RQ2) reveal that there are indeed certain differences among user emotions across different contact tracing apps introduced by India and Singapore. We also found under what scenarios such differences exist: TraceTogether has garnered more positive emotions than AarogyaSetu around keywords such as security, tracing, risk, protection and sharing. This revelation prompts further investigation on why such differences exist. Cultural differences across the countries, differences in privacy-related features of the apps themselves and/or differences in each government’s attitude towards privacy could possibly explain the differences observed in user emotions. By answering RQ2, we also show that our proposed methodology can be quickly extended to cover multiple countries and cultures. Keyword Tweet security Lovely to see sensible information security and a very clear explanation in the Singapore TraceTogether app https://t.co/yf5CQUyvUU risk And the benefits of many people using #TraceTogether outweigh the possible risk it might hold. COVID-19 is a big deal. location The TraceTogether app can identify people who have been within 2m of coronavirus patients for at least 30 minutes, using wireless Bluetooth technology. It does not collect or use location data. https://t.co/i09tiiZcR3 Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 24 6 Practical Implications Our study has important practical implications for data controllers who determine the purposes for which and the means by which personal data is processed. As subsequent waves of the pandemic hit different nations incessantly and the possibility of other pandemics in the future where contact tracing may become necessary, it is important that data controllers pay close attention to users’ emotional responses. This could be the first step in understanding users’ privacy-related perceptions around the app. We show that potential users of contact tracing apps have expressed extensive negative emotions around privacy harms related to these apps. Negative emotions can significantly influence users’ privacy protection beliefs and perceived privacy risks (Li et al., 2011). According to Privacy Calculus Theory, negative beliefs of privacy concerns may in turn lower chances of adoption of contact tracing apps. It is equally important for data controllers to understand what leads to these emotional responses. Our study shows higher positive emotional responses such as happiness when users perceived a higher level of privacy protection and negative emotions such as fear when they perceived loss of control over data and insufficient protection measures. Users’ positive appraisals about their experience interacting with an app (motive consistency) can trigger liking emotions (Li et al., 2017). High levels of uncertainty, such as how data would be processed and shared in the absence of a comprehensive data protection regulation, are strongly associated with fear (for example, fear of surveillance) (Watson & Spence, 2007). Similarly, accidental, negligent and intentional harm that is attributable to a known person(s), such the feeling that the government is responsible for not doing enough to protect personal data collected by the app, may lead to anger and outrage (Watson & Spence, 2007). Based on these insights, data controllers can take one or more of the following actions to reduce negative emotions and fuel positive ones to improve the uptake of the app. 1) Data controllers can focus on improving the overall experience in interacting with the app and the extent of privacy control including the availability of layered privacy policy to trigger positive emotions (Li et al., 2017). 2) The introduction and enforcement of strong regulatory measures governing data processing and increasing transparency regarding the app’s privacy features can possibly lead to lower negative emotions from agency as well as uncertainty perspectives. 3) The government could also perform Data Protection Impact Assessment (DPIA) for the app to minimize users’ privacy risks and publish the DPIA report to improve stakeholders’ confidence in the app and improve its own accountability. Further, we found differences in emotional reactions to different contact tracing apps. Understanding these differences can help data controllers to comprehend the effects of privacy-related cultural characteristics of its user base on their emotional responses and fathom what types of app privacy features and protective measures are appreciated by users and which privacy risks lead to negative reactions. 7 Conclusion This research provides insights regarding user emotions towards privacy aspects of contact tracing apps using social media analysis. It shows that users express negative emotions of Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 25 anger and fear when they perceive privacy risks or when they perceive a lack of privacy protection measures. In many cases, such negative emotions have resulted from negative press. Similarly, citizens express positive emotions when they feel satisfied about privacy protection measures. The study also shows that more negative emotions have been expressed towards the Indian contact tracing app AarogyaSetu compared to the Singaporean app TraceTogether. This research presents an effective way of analysing UGC in the context of privacy and compares emotions towards two contact tracing apps. However, there remains a lack of efforts in examining UGC to analyse privacy related emotions towards e-government products and platforms, in general, as well as cross-cultural examination of privacy emotions. Future studies can therefore focus on understanding users’ privacy related emotions towards e-government initiatives, establish possible connections between such emotions and the adoption of such initiatives by users and compare privacy emotions towards e-government initiatives globally. The study indicated differences in privacy emotions towards the two apps, however, it did not engage in a rigorous quantitative analysis to validate the same. Although Twitter provides suitable user data, a lot of this data remains unexplored due to the limited possibilities to extract the data through APIs. The study only explored tweets in English, excluding all other language data which may have led to overlooking of significant information related to what users feel about privacy of contact tracing apps. The present study has only examined Twitter data. However, users may express their opinions on other SM platforms as well as app review forums and Google/Apple play stores. Since the study only explores Twitter posts, an element of selection bias could be involved, where people who have a negative opinion are more likely to tweet it than people who are happy. Future research could address these issues. Acknowledgement We thank the section editor and anonymous reviewers for their valuable suggestions and guidance to improve our article in each stage of revision. References Anderson, C.L. & Agarwal, R. (2011) The digitization of healthcare: boundary risks, emotion, and consumer willingness to disclose personal health information. Information Systems Research, 22, 469–490. Arnold, M. B. (1960). Emotion and personality (Vol. I & II). New York. Columbia University Press. Azad, M. A., Arshad, J., Akmal, S. M. A., Riaz, F., Abdullah, S., Imran, M., & Ahmad, F. (2020). A first look at privacy analysis of COVID-19 contact tracing mobile applications. IEEE Internet of Things Journal. https://doi.org/10.48550/arXiv.2006.13354 Balapour, A., Nikkhah, H. R., & Sabherwal, R. (2020). Mobile application security: Role of perceived privacy as the predictor of security perceptions. International Journal of Information Management, 52, 102063. Barnard, L. (2014). The cost of creepiness: How online behavioral advertising affects consumer purchase intention (Doctoral dissertation, The University of North Carolina at Chapel Hill). Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 26 Baumgärtner, L., Dmitrienko, A., Freisleben, B., Gruler, A., Höchst, J., Kühlberg, J., ... & Uhl, C. (2020). Mind the GAP: Security & privacy risks of contact tracing apps. In 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), 458-467. Beldad, A., De Jong, M., & Steehouder, M. (2011). I trust not therefore it must be risky: Determinants of the perceived risks of disclosing personal data for e-government transactions. Computers in Human Behavior, 27(6), 2233-2242. Bengio, Y., Ippolito, D., Janda, R., Jarvie, M., Prud'homme, B., Rousseau, J. F., ... & Yu, Y. W. (2021). Inherent privacy limitations of decentralized contact tracing apps. Journal of the American Medical Informatics Association, 28(1), 193-195. Bollen, J., Mao, H., & Pepe, A. (2011). Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena. In Proceedings of the International AAAI Conference on Web and Social Media, 5(1). Buchanan, T., Paine, C., Joinson, A. N., & Reips, U. D. (2007). Development of measures of online privacy concern and protection for use on the Internet. Journal of The American Society for Information Science and Technology, 58(2), 157-165. Chan, E. Y., & Saqib, N. U. (2021). Privacy concerns can explain unwillingness to download and use contact tracing apps when COVID-19 concerns are high. Computers in Human Behavior, 119, 106718. Cho, H., Ippolito, D., & Yu, Y. W. (2020). Contact tracing mobile apps for COVID-19: Privacy considerations and related trade-offs. arXiv preprint arXiv:2003.11511. Church, K., & Hanks, P. (1990). Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1), 22-29. Clore, G. L., Gasper, K., & Garvin, E. (2001). Affect as information. Handbook of Affect and social Cognition, 121-144. Routledge, London, UK. Crable, E., & Sena, M. (2020). Exploring Sentiment Towards Contact Tracing. In Proceedings of the Conference on Information Systems Applied Research, Vol. 2167, p. 1508. Culnan, M. J., & Armstrong, P. K. (1999). Information privacy concerns, procedural fairness, and impersonal trust: An empirical investigation. Organization Science, 10(1), 104-115. Damasio, A. R. (1994). Descartes’ error: Emotion, rationality and the human brain. Putman, New York, USA. Dinev, T., & Hart, P. (2006). An extended privacy calculus model for e-commerce transactions. Information Systems Research, 17(1), 61-80. Dinev, T., Hart, P., & Mullen, M. R. (2008). Internet privacy concerns and beliefs about government surveillance–An empirical investigation. The Journal of Strategic Information Systems, 17(3), 214-233. Ekman P. (2007). Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life. Holt, New York, USA. Ekman, P. (1992). An argument for basic emotions. Cognition & Emotion, 6(3-4), 169-200. Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 27 Fahey, R. A., & Hino, A. (2020). COVID-19, digital privacy, and the social limits on data- focused public health responses. International Journal of Information Management, 55, 102181. Fan, W., & Gordon, M. D. (2014). The power of social media analytics. Communications of the ACM, 57(6), 74-81. Featherman, M. S., & Pavlou, P. A. (2003). Predicting e-services adoption: a perceived risk facets perspective. International Journal of Human-Computer Studies, 59(4), 451-474. Featherman, M. S., Miyazaki, A. D., & Sprott, D. E. (2010). Reducing online privacy risk to facilitate e-service adoption: the influence of perceived ease of use and corporate credibility. Journal of Services Marketing, 24(3), 219-229 Fiesler, C., & Hallinan, B. (2018). "We Are the Product" Public Reactions to Online Data Sharing and Privacy Controversies in the Media. In Proceedings of the 2018 CHI Conference on Human Factors In Computing Systems, 1-13. Fox, G., Clohessy, T., van der Werff, L., Rosati, P., & Lynn, T. (2021). Exploring the competing influences of privacy concerns and positive beliefs on citizen acceptance of contact tracing mobile applications. Computers in Human Behavior, 121, 106806. Frijda, N. H. (1994). Varieties of affect: Emotions and episodes, moods, and sentiments. In P. Ekman & R. J. Davidson (Eds.), The nature of emotion (pp. 59–67). New York: Oxford University Press. Georgieva, I., Beaunoyer, E., & Guitton, M. J. (2021). Ensuring social acceptability of technological tracking in the COVID-19 context. Computers in Human Behavior, 116, 106639. Giachanou, A., & Crestani, F. (2016). Like it or not: A survey of twitter sentiment analysis methods. ACM Computing Surveys (CSUR), 49(2), 1-41. González, F., Figueroa, A., López, C., & Aragon, C. (2019a, November). Information Privacy Opinions on Twitter: A Cross-Language Study. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing,190-194. González, F., Yu, Y., Figueroa, A., López, C., & Aragon, C. (2019b, May). Global reactions to the Cambridge analytica scandal: A cross-language social media study. In Companion Proceedings of the 2019 World Wide Web Conference, 799-806. Greene, J., & Haidt, J. (2002). How (and where) does moral judgment work?. Trends in Cognitive Sciences, 6(12), 517-523. Gu, J., Xu, Y. C., Xu, H., Zhang, C., & Ling, H. (2017). Privacy concerns for mobile app download: An elaboration likelihood model perspective. Decision Support Systems, 94, 19-28. Gutierrez, A., O'Leary, S., Rana, N. P., Dwivedi, Y. K., & Calle, T. (2019). Using privacy calculus theory to explore entrepreneurial directions in mobile location-based advertising: Identifying intrusiveness as the critical risk factor. Computers in Human Behavior, 95, 295-306. Horvath, L., Banducci, S., & James, O. (2020). Citizens’ attitudes to contact tracing apps. Journal of Experimental Political Science, 1-13. Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 28 Jarvenpaa, S. L., Tractinsky, N., & Vitale, M. (2000). Consumer trust in an Internet store. Information Technology and Management, 1(1), 45-71. Johnson, R., & Zhang, T. (2015). Effective use of word order for text categorization with convolutional neural networks. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2015 (p. 103). Kehr, F., Kowatsch, T., Wentzel, D., & Fleisch, E. (2015). Blissfully ignorant: the effects of general privacy concerns, general institutional trust, and affect in the privacy calculus. Information Systems Journal, 25(6), 607-635. Kehr, F., Wentzel, D., & Mayer, P. (2013). Rethinking the Privacy Calculus: On the Role of Dispositional Factors and Affect. In proceedings of the thirty-fourth International Conference on Information Systems, 1-10. Keith, M. J., Thompson, S. C., Hale, J., Lowry, P. B., & Greer, C. (2013). Information disclosure on mobile devices: Re-examining privacy calculus with actual user behavior. International Journal of Human-Computer Studies, 71(12), 1163-1173. Keltner, D., Lerner J. S. (2010). Emotion. In The Handbook of Social Psychology, Vol. 1, ed. Gilbert, D. T., Fiske, S. T., Lindzey, G., pp. 317–52. Wiley, Hoboken, NJ, USA. Keltner, D., Oatley, K., & Jenkins, J. M. (2014). Understanding emotions. Wiley, Hoboken, NJ, USA. Kim, D. J., Ferrin, D. L., & Rao, H. R. (2009). Trust and satisfaction, two stepping stones for successful e-commerce relationships: A longitudinal exploration. Information Systems Research, 20(2), 237-257. Leith, D. J., & Farrell, S. (2020, October). Coronavirus contact tracing app privacy: What data is shared by the Singapore OpenTrace app? In International Conference on Security and Privacy in Communication Systems, 80-96. Cham, Switzerland: Springer. Lerner, J. S., Li, Y., Valdesolo, P., & Kassam, K. S. (2015). Emotion and decision making. Annual Review of Psychology, 66, 799-823. Li, H., Luo, X. R., Zhang, J., & Xu, H. (2017). Resolving the privacy paradox: Toward a cognitive appraisal and emotion approach to online privacy behaviors. Information & Management, 54(8), 1012-1022. Li, H., Sarathy, R., & Xu, H. (2011). The role of affect and cognition on online consumers' decision to disclose personal information to unfamiliar online vendors. Decision Support Systems, 51(3), 434-445. Li, H., Sarathy, R., & Zhang, J. (2008). The role of emotions in shaping consumers’ privacy beliefs about unfamiliar online vendors. Journal of Information Privacy and Security, 4(3), 36-62. Li, Y. (2012). Theories in online information privacy research: A critical review and an integrated framework. Decision Support Systems, 54(1), 471-481. Li, Y. (2014). The impact of disposition to privacy, website reputation and website familiarity on information privacy concerns. Decision Support Systems, 57, 343-354. Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 29 Lin, J., Carter, L., & Liu, D. (2021). Privacy concerns and digital government: exploring citizen willingness to adopt the COVIDSafe app. European Journal of Information Systems, 30(4), 1-14. Loewenstein, G. F., Weber, E. U., Hsee, C. K., & Welch, N. (2001). Risk as feelings. Psychological Bulletin, 127(2), 267. Malhotra, N. K., Kim, S. S., & Agarwal, J. (2004). Internet users' information privacy concerns (IUIPC): The construct, the scale, and a causal model. Information Systems Research, 15(4), 336-355. Masur, P. K., & Scharkow, M. (2016). Disclosure management on social network sites: Individual privacy perceptions and user-directed privacy strategies. Social Media+ Society, 2(1), 2056305116634368. Mohammad, S. (2012). # Emotional tweets. In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics–Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), 246-255). Pak, A., & Paroubek, P. (2010, May). Twitter as a corpus for sentiment analysis and opinion mining. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC 2010), 1320-1326. Pang, B., & Lee, L. (2009). Opinion mining and sentiment analysis. Computational Linguistics, 35(2), 311-312. Peng, H., Li, J., He, Y., Liu, Y., Bao, M., Wang, L., Song, Y., & Yang, Q., 2018, April. Large- scale hierarchical text classification with recursively regularized deep graph-cnn. In Proceedings of the 2018 World Wide Web Conference (pp. 1063-1072). Praveen, S. V., & Ittamalla, R. (2021). Analyzing Indian citizen's perspective towards government using wearable sensors to tackle COVID-19 crisis—A text analytics study. Health Policy and Technology, 10(2), 100521. Praveen, S. V., Ittamalla, R., & Subramanian, D. (2020a). Challenges in successful implementation of Digital contact tracing to curb COVID-19 from global citizen’s perspective: A text analysis study. International Journal of Pervasive Computing and Communications, 1-8. Praveen, S. V., Ittamalla, R., & Subramanian, D. (2020b). How optimistic do citizens feel about digital contact tracing? –Perspectives from developing countries. International Journal of Pervasive Computing and Communications, 1-9. Raber, F., & Krüger, A. (2018, July). Privacy perceiver: Using social network posts to derive users' privacy measures. In Adjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization, 227-232. ACM, New York, USA. Rathore, A. K., & Ilavarasan, P. V. (2020). Pre-and post-launch emotions in new product development: Insights from twitter analytics of three products. International Journal of Information Management, 50, 111-127. Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 30 Rathore, A. K., Maurya, D., & Srivastava, A. K. (2021). Do policymakers use social media for policy design? A Twitter analytics approach. Australasian Journal of Information Systems, 25. https://doi.org/10.3127/ajis.v25i0.2965 Rekanar, K., O’Keeffe, I. R., Buckley, S., Abbas, M., Beecham, S., Chochlov, M., ... & Buckley, J. (2021). Sentiment analysis of user feedback on the HSE’s Covid-19 contact tracing app. Irish Journal of Medical Science, 191(1), 103-112. Rowe, F. (2020). Contact tracing apps and values dilemmas: A privacy paradox in a neo- liberal world. International Journal of Information Management, 55, 102178. Shaw, N., & Sergueeva, K. (2019). The non-monetary benefits of mobile commerce: Extending UTAUT2 with perceived value. International Journal of Information Management, 45, 44- 55. Simko, L., Chang, J. L., Jiang, M., Calo, R., Roesner, F., & Kohno, T. (2020). COVID-19 contact tracing and privacy: A longitudinal study of public opinion. arXiv preprint arXiv:2012.01553, 1-37. Singh, J. P., Dwivedi, Y. K., Rana, N. P., Kumar, A., & Kapoor, K. K. (2019). Event classification and location prediction from tweets during disasters. Annals of Operations Research, 283(1), 737-757. Smith, C. A. & Kirby, L. (2000) Consequences require antecedents: Toward a process model of emotion elicitation. In Forgas, J. P. (Ed.) Feeling and Thinking: The role of affect in social cognition. Cambridge University Press. Stieglitz, S., Dang-Xuan, L., Bruns, A., & Neuberger, C. (2014). Social media analytics-an interdisciplinary approach and its implications for information systems. Business & Information Systems Engineering, 6(2), 89-96. Stieglitz, S., Mirbabaie, M., Ross, B., & Neuberger, C. (2018). Social media analytics– Challenges in topic discovery, data collection, and data preparation. International Journal of Information Management, 39, 156-168. Sun, Y., Wang, N., Shen, X. L., & Zhang, J. X. (2015). Location information disclosure in location-based social network services: Privacy calculus, benefit structure, and gender differences. Computers in Human Behavior, 52, 278-292. Tsytsarau, M., & Palpanas, T. (2012). Survey on mining subjective data on the web. Data Mining and Knowledge Discovery, 24(3), 478-514. Ur, B., & Wang, Y. (2013, May). A cross-cultural framework for protecting user privacy in online social media. In Proceedings of the 22nd International Conference on World Wide Web, 755-762. ACM, New York, USA. Ur, B., Leon, P. G., Cranor, L. F., Shay, R., & Wang, Y. (2012, July). Smart, useful, scary, creepy: perceptions of online behavioral advertising. In Proceedings of The Eighth Symposium on Usable Privacy and Security, 1-15. ACM, New York, USA. Wakefield, R. (2013). The influence of user affect in online information disclosure. The Journal of Strategic Information Systems, 22(2), 157-174. Australasian Journal of Information Systems Mehta, De & Chattopadhyay 2022, Vol 26, Research Article Elucidating the role of emotion in privacy-concerns 31 Walrave, M., Waeterloos, C., & Ponnet, K. (2020). Adoption of a contact tracing app for containing COVID-19: a health belief model approach. JMIR Public Health and Surveillance, 6(3), e20572, 1-10. Watson, L., & Spence, M. T. (2007). Causes and consequences of emotions on consumer behaviour: A review and integrative cognitive appraisal theory. European Journal of Marketing, 41(5/6), 487-511. Wen, H., Zhao, Q., Lin, Z., Xuan, D., & Shroff, N. (2020, October). A study of the privacy of covid-19 contact tracing apps. In International Conference on Security and Privacy in Communication Systems, 297-317. Springer, Cham, Switzerland. Wigan, M. (2020). Rethinking IT Professional Ethics: Classical and Current Contexts. Australasian Journal of Information Systems, 24. https://doi.org/10.3127/ajis.v24i0.2851 Wildenauer, M. (2020). The Shared Responsibility Model: Levers of Influence and Loci of Control to aid Regulation of Ethical Behaviour in Technology Platform Companies. Australasian Journal of Information Systems, 24. https://doi.org/10.3127/ajis.v24i0.2797 Xu, H., Luo, X. R., Carroll, J. M., & Rosson, M. B. (2011). The personalization privacy paradox: An exploratory study of decision making process for location-aware marketing. Decision Support Systems, 51(1), 42-52. Xu, H., Teo, H. H., & Tan, B. C. (2005). Predicting the adoption of location-based services: The role of trust and perceived privacy risk. In 26th International Conference on Information Systems, ICIS 2005 (pp. 897-910). Zeng, D., Chen, H., Lusch, R., & Li, S. H. (2010). Social media analytics and intelligence. IEEE Intelligent Systems, 25(6), 13-16. Zhang, B., & Xu, H. (2016, February). Privacy nudges for mobile applications: Effects on the creepiness emotion and privacy attitudes. In Proceedings of the 19th ACM conference on computer-supported cooperative work & social computing (pp. 1676-1690). Copyright: © 2022 authors. This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial 3.0 Australia License, which permits non- commercial use, distribution, and reproduction in any medium, provided the original author and AJIS are credited. doi: https://doi.org/10.3127/ajis.v26i0.3687 http://creativecommons.org/licenses/by-nc/3.0/au/ 1 Introduction 2 Theoretical Background and Review of Literature 2.1 Emotions, Decision-making and Privacy 2.2 Overview on Privacy Calculus Theory 2.3 Overview on Perceived privacy risk 2.4 Overview on perceived privacy protection 2.5 Social Media Analytics in Opinion Mining and Sentiment Analysis 2.6 Gaps in extant literature and the contributions of this research 3 Method and Data 3.1 Creation of Target Datasets 3.2 Creation of Auxiliary Dataset 3.3 Proposed Method 3.4 Experimental Performance 4 Results and Discussion 4.1 Emotion Analysis of AarogyaSetu Tweets 4.2 Emotion Analysis of TraceTogether Tweets 5 Theoretical Contribution 6 Practical Implications 7 Conclusion