Vol8No3Paper4Svarre and Gaardboe To cite this article: Svarre, T and Gaardboe, R. (2018) Characterizing business intelligence tasks, use and users in the workplace. Journal of Intelligence Studies in Business. 8 (3) 45-54. Article URL: https://ojs.hh.se/index.php/JISIB/article/view/328 This article is Open Access, in compliance with Strategy 2 of the 2002 Budapest Open Access Initiative, which states: Scholars need the means to launch a new generation of journals committed to open access, and to help existing journals that elect to make the transition to open access. Because journal articles should be disseminated as widely as possible, these new journals will no longer invoke copyright to restrict access to and use of the material they publish. Instead they will use copyright and other tools to ensure permanent open access to all the articles they publish. Because price is a barrier to access, these new journals will not charge subscription or access fees, and will turn to other methods for covering their expenses. There are many alternative sources of funds for this purpose, including the foundations and governments that fund research, the universities and laboratories that employ researchers, endowments set up by discipline or institution, friends of the cause of open access, profits from the sale of add-ons to the basic texts, funds freed up by the demise or cancellation of journals charging traditional subscription or access fees, or even contributions from the researchers themselves. There is no need to favor one of these solutions over the others for all disciplines or nations, and no need to stop looking for other, creative alternatives. Journal of Intelligence Studies in Business Publication details, including instructions for authors and subscription information: https://ojs.hh.se/index.php/JISIB/index Characterizing business intelligence tasks, use and users in the workplace Tanja Svarrea* and Rikke Gaardboea aAalborg Universitet, Denmark; *tanjasj@hum.aau.dk Journal of Intelligence Studies in Business PLEASE SCROLL DOWN FOR ARTICLE Characterizing business intelligence tasks, use and users in the workplace Tanja Svarrea* and Rikke Gaardboea a Aalborg Universitet, Denmark Corresponding author (*): tanjasj@hum.aau.dk Received 14 June 2018 Accepted 27 December 2018 ABSTRACT This paper investigates business intelligence (BI) tasks, use and users in a workplace setting. The study reports on a mixed methods study of users in three different types of organisations employing BI. 1052 respondents answered a survey and 15 individual and 3 group interviews were conducted to elaborate on the survey results. The study finds that the majority of public BI users are employees, and fewer managers and students, that are handling a variety of tasks. Although they can experience challenges learning and using the BI system, they are still satisfied with it from different perspectives. KEYWORDS Business intelligence, information system use, workplace studies 1. INTRODUCTION In 2017, Gartner performed a worldwide survey of IT spending among 2500 chief information officers (CIOs). Business intelligence (BI) was one of the top technology priorities they identified (“Gartner Survey of More Than 2,500 CIOs Charts the Rise of the Digital Ecosystem,” n.d.). One of the reasons for this focus on BI can be attributed to the increasing importance of BI systems. BI can be defined as "a broad category of technologies, applications, and processes for gathering, storing, accessing, and analysing data to help its users make better decisions" (Wixom & Watson, 2010, p. 13). In recent years, BI technologies have received considerable attention from both industry and the public sector (Chen et al. 2012). BI is an interesting technology because several studies have shown that there is a relationship between computer- driven decisions and organisational performance (Brynjolfsson, Hitt, & Kim, 2011). However, achieving BI success depends on both organization and staff characteristics (Worley et al. 2005; Salmasi et al. 2016). Salmasi et al. (2016) previously conducted a study of organisational level competences in achieving success with BI. However, García and Pinzón (2017) found that amongst others, the human perspective along with learning and skills are highly important to success. Therefore, we will focus on the individual perspective and focus on the users in this paper. At the same time as the development of government processes, organisations and technologies are expected to change government employees' tasks. Before the emergence of e-government, governments’ information technology and data management tasks were largely related to employment (see Kraemer & Dedrick, 1997). Today, changes in work tasks are an expected consequence of governments’ digitising efforts. In particular, e-government is expected to affect the composition of public employees' tasks (Dörfler, 2003; Snellen, 2002). Jürgensen (2012) documented employees' expectations within the framework of administrative grants and found that specific, routine tasks had fallen from employees’ daily tasks while the proportion of challenging applications had Journal of Intelligence Studies in Business Vol. 8, No. 3 (2018) pp. 45-54 Open Access: Freely available at: https://ojs.hh.se/ 46 risen. Others note similar findings regarding tax department employees. The present paper is concerned with characterising the tasks users solve with BI. Thus, to achieve success with BI at the user level, there is a relationship between task characteristics and success (Petter et al. 2013). Furthermore, we examine BI systems’ ability to underpin the tasks their users solve with BI in a Danish e-government setting. The use of BI in e-government has spread worldwide in the latest decade. By improving access to BI among employees, governments are aiming to improve their decision-making processes, resource use, increase quality of the services delivered or even reduce costs. The paper is structured as follows: In the following section, we review existing studies of task characteristics and present the theoretical framework for the data collection process. The next section is a presentation of the research methods applied in the study: a survey questionnaire, 12 semi-structured interviews and three group interviews. The subsequent section presents our findings with regard to the research question. The paper concludes with closing remarks and suggestions for further research. 2. THEORETICAL BACKGROUND 2.1 The concept of a task A task is what individuals engage in to keep their work or life continuing (Li & Belkin, 2008), and the concept of ‘tasks’ is important in human–computer interaction. A task (whether work- or leisure-related) may trigger information-oriented activities (Byström Katriina & Hansen Preben, 2005). On this basis, the task becomes a central element of any user’s context, as it arises from an incident external to the user that first triggers an information need, followed by a searching activity (Ingwersen & Järvelin, 2005). Being external to the user, the task, as such, will be easier to observe and measure, from a research perspective, if compared to information needs that are inherent to the user. Tasks have been analysed in human– computer interactions from many different perspectives. Historically, the focus on tasks went from a technical (ergonomic) perspective, to a conceptual (information processing) perspective and then to work-process (contextual) models (Crystal & Ellington, 2004). Different approaches can be followed to gain insight into tasks in specific contexts. Hierarchical task analysis breaks generic tasks into smaller sub tasks with related sub goals. (Stanton, 2006). The purpose is to become able to map goals, and sub goals in particular, with technologies or information systems to ensure successful solutions for the users’ tasks. A different way of perceiving tasks is to model them according to Li and Belkin’s (Li & Belkin, 2008) taxonomy of task characteristics. Departing from a literature review, the taxonomy defines tasks on the basis of generic facets and common attributes, thus representing a top-down perspective on the task concept. The different approaches to understanding and operationalising tasks emphasises the importance of the concept in human–computer interaction. We have not identified any papers to date within the BI systems field that have attempted to identify and characterise the specific tasks users carry out. The purpose of the current paper is to address this gap in the research in a public organisation context. 2.2 DeLone & McLean: The IS Success Model DeLone and McLean’s IS success model (DeLone & McLean, 1992) is used to frame the study. The model represents a framework for understanding influential factors on information systems’ success. The identified variables in the model include system quality, information quality, use, user satisfaction, individual impact and organisational impact. We use the model to frame the quantitative data collection below, as it represents a consolidated theoretical model (eg. Iivari, 2005), providing both an organisational and system-based perspective on the notion of tasks. 3. RESEARCH DESIGN AND METHOD In this study, we used a multiphase ‘mixed methods’ research design. The research design consists of two main phases, namely a questionnaire and interviews. The mixed methods approach represents a form of triangulation; the quantitative approach provides a broader view, while the qualitative approach provides greater depth. Together, the approaches yield results from which more accurate inferences can be made (Seddon et al. 1999). 3.1 Quantitative method We chose a questionnaire to research users' perceptions of different task characteristics. 47 Data were collected via an online survey available for a specific period during the spring of 2017. All respondents were BI end-users who had access rights to their organisation’s BI system. The users accessed the BI web client through a browser, meaning that BI can be implemented across an entire organisation without having to install software on each machine. All BI users from three public organisations were invited to complete the survey. The three organisations were a municipality using Business Objects, a public healthcare organisation (among 12 hospitals) using Tableau and a university using QlikView. Initially, we conducted a pilot study before distributing the survey to all invitees. The survey was based on a literature review, and three researchers in the field evaluated the questions. Afterwards, BI users with differing levels of BI experience evaluated the questionnaire using a think-aloud test (Nielsen, 1994). Minor refinements were made based on these results. The final part of the pilot study called for testing the survey on 24 BI users. After evaluating those results, the questionnaire was distributed by email to 4901 invitees. Participants accessed the questionnaire via a personal invitation email with a unique link to the online survey. Each respondent received an adapted questionnaire depending on whether he or she had previously used or never used the BI in question. Participation in the survey was voluntary, and two reminders were sent. In total, 1741 people completed the survey, resulting in a response rate of 35.52%. Among these, 1052 were used for the statistical analysis, as 689 respondents indicated that they did not use BI. All data were analysed in SPSS version 24.0. 3.2 Qualitative method The next step in our research design was interviewing BI users. In addition to the questionnaire, we used interviews for three reasons: qualitative data can explain the complexity of the users’ tasks identified in the survey, data from interviews helps us to grasp the users’ contexts and interviews make it possible to check for potential additional elements of BI systems’ successes or failures (Driscoll et al. 2007). We conducted 15 interviews as part of the qualitative study, and three group interviews with a total of seven participants were arranged for the three organisations. The results from the survey were presented to the groups, and the participants in the group interviews commented on the survey results. Afterwards, we formulated a semi-structured interview. The semi-structured interviews had an average length of 45 minutes. All interviews were transcribed and analysed in NVivo version 11.0. We used a deductive method to categorise the different tasks’ descriptions. The different categories were adapted from earlier work. We will use the interviews to exemplify quantitative findings in the analysis. 4. RESULTS The results of the study are presented in three sections: end users’ characteristics, task characteristics and the users’ assessments of BI success. 4.1 End user characteristics In the survey, the respondents were asked about their gender, age (Table 1), education, organisational role and experience. Table 1 Respondents' ages. Age N % 20–29 years 66 4 30–39 years 225 22 40–49 years 345 33 50–59 years 325 31 60–69 years 91 9 Total 1052 100 Table 2 Organizational roles. Role N % Employees 758 72 Managers 223 22 Students 65 6 Missing 6 0 Total 1052 100 Most of the respondents were women (73%). As shown in Table 1, the majority of respondents were 40–49 years old. Their educational levels varied; most commonly, the respondents either held a master’s degree (35%) or a vocational degree (30%) (see Figure 1). The respondents’ organisational roles were distributed among employees (72%), managers 48 (21%) and students (6%) (see Table 2). For the sake of comparison, Negash & Gray (2008) found that BI is mainly used by managers and highly educated employees. Finally, we can characterise the respondents in terms of their BI experience. The distribution appears in Figure 2. The question asked the respondents to assess their BI experience on a scale from 1 to 5, where 1 is ‘little experience’ and 5 is ‘great experience’. More than three-fourths (76%) of the respondents rated their BI experience at 3 or below, indicating that they were not highly experienced users. Here, it should be noted that two of the three organisations under investigation had used BI for a number of years, while one implemented BI about two years ago (in 2016). This difference of time spent with BI may explain some of the differences in experience assessments among the organisations. In related studies, technology experience has been found to be a critical factor for system success (Dishaw & Strong, 2003; Marshall et al. 2000; Thompson et al. 1994). Corresponding explanations were found in the interviews. One interview participant explained: “…the more experience you get with the system, the more you think: ‘Well, this is fine and really easy to understand’. But then when you get out and have to explain it – for instance at meetings in our controller group, if I have prepared something and ask ‘What do you think about this?’, then they are like, ‘We don’t understand that’, and I think, ‘Well, that is easy to understand’. But you easily get into an understanding of what you think is easy to understandable” (2017). Apart from confirming the importance of experience, the quote above also illustrates the difference between system users (users interacting with the system) and information users (employees using the information from BI). 4.2 Characteristics of BI tasks We identified BI tasks from several different dimensions in the study. At an overall level, the respondents were asked what BI was primarily used for. The distribution of their answers is shown in Figure 3. More than half (56%) reported their main use is for data extraction, 29.8% point to reporting and the last 14.2% mentioned ad-hoc analysis as their most frequent use of the system. The respondents were also asked what specific BI functionalities they use. The results appear in Figure 4. As shown, the most-used function by far was data filtering, followed by compiling data in a table and visualisation. Less common functionalities included drilling down, layout formatting, calculations (e.g., numeration) and merging (e.g., linking data together from different sources). Figure 1 Respondents’ educational levels. 0 50 100 150 200 250 300 350 1 2 3 4 5 Figure 2 Respondents’ assessments of their BI experience 14% 56% 30% Ad hoc-analysis Data extraction Reporting Figure 3 What is BI primarily used for? 49 Figure 4 Functionalities used in BI Figure 5 The amount of work tasks in which BI is used Figure 5 presents the share of tasks that BI represents among the total tasks handled by the respondents. As is evident from the figure, the majority use BI less than half of the time, or not at all. Thus, BI use represents a minor part of the total number of tasks respondents handled. However, based on interview data, it appears that, despite these minor use patterns, users still consider BI to be an important tool in their everyday work practice. All statements in the survey were rated on a 5-point scale, with 1 being highly disagree and 5 being highly agree. As shown in Table 3, the mean response rating is above 3, indicating that the respondents more or less agree with the statement. The statement with the lowest rating addresses the amount of data in the system and the relation with the respondents’ tasks. In the interviews, more participants claimed that they think the amount of data is appropriate. One comment may explain some of the lower rating of the statement. The participant states: “As regards the report module, I can create the things I would like to, but it is less appropriate in terms of publication and dissemination. It comes in short in terms of saying ’We would like to continue here, but we can’t with this tool, so we need new technology to move on’” (2017). Table 3 Univariate statistics on ‘Task compatibility’ Reply Min Max Mean SD This information is useful for my work 1 5 3.86 0.974 This information is complete for my needs 1 5 3.28 0.976 This information is sufficiently up-to-date for my work 1 5 3.46 1.04 This information is relevant to our work 1 5 3.45 0.943 Table 4 Univariate statistics on ‘Task significance’ Statement Min Max Mean SD The tasks I complete in BI are an important part of my tasks. 1 5 3.44 1.180 I make decisions on the basis of the tasks I complete in BI. 1 5 3.32 1.284 My tasks completed in BI are important to other employees in the organisation. 1 5 3.50 1.176 Other people make decisions based on the tasks I complete in BI. 1 5 3.45 1.234 My tasks in BI are important for collaborators outside the organisation. 1 5 2.28 1.273 Error! Reference source not found. shows the distribution of responses as regards ‘Task significance’. In all statements, except for the last one, the mean value is between 3 and 4. In general, the respondents consider their BI tasks to be important, and they or others make decisions on these tasks. The respondents do not consider the tasks to be important for collaboration outside of their organisation, so the BI is instead used as an internal tool. The following quotes from the interviews illustrate the significance of BI tasks: “The tasks are pivotal, because we need to touch upon the economy so much. We need to file reports very, very much” (2018). “I think it is quite important, at least in relation to many of the requests we get. We get a lot of requests that are used politically, or […] 773 434 308 181 170 126 100 57 0 100 200 300 400 500 600 700 800 900 Fi lte r d ata Me rg e d ata in a… Vi su ali za tio n Dr ill- do wn No t u sin g t he … Fo rm att ing la yo ut Ma ke ca lcu lat ion s Me rg e d ata fr om … 16% 70% 7%6%1% None Below half part of the tasks Half part of the tasks Above half part of the tasks All tasks 50 So, if we didn’t have the option […]. Usually, it is with a very short time frame, where a politician asks, ‘We need this for our…’ Or it can be on the same day that you are in a meeting and they go, ‘We need this…’ and that would then be within an hour” (2017). The quotes illustrate why BI is important to the users. One thing is that they need to file reports within their organisations. The other is that several users receive requests from others regarding facts that are being drawn from the BI system. Table 5 Univariate statistics on ‘Task interdependence’. Statement Min Max Mean SD If I do not complete my tasks in BI, one or more employees in the organisation cannot complete their tasks. 1 5 2.49 1.346 In BI, I can only do tasks if one or more employees have completed another task first. 1 5 2.61 1.378 I am independent of other employees to prepare tasks in BI. 1 5 2.94 1.339 Task interdependence reflects the users’ dependencies in relation to the system; this can be in terms of a user’s dependence on something, or another’s dependence on the user. These assessments are presented in Table 5. Here, we can see that, across all three organisations, the users do not depend on anything to use BI themselves. It is assumed that the BI system is available, updated and so on. Further, the respondents disagree that their tasks depend on colleagues’ completion of other tasks first. Although the ratings for task interdependence are low, the interviews revealed dependencies, typically in the participants’ ability to deliver information to other employees, such as managers. To illustrate: “Well, the closest managers” (2017). “The department management, and then our doctors. They are the ones using me for this” (2017). “That would typically be our political committees or the management of our administration” (2017). A partial explanation for the dependencies reported of the respondents may be found in this quote from the interviews: “Everyone can go in and get data. It is just not everyone [who] know[s] how to use it. The benefit of asking me is that I know data better than most people, and by that I also know how to use data and how to do this. That’s how it works” (2017). In sum, one of the barriers to employees’ access to the BI system is a lack of knowledge of the underlying data models. The respondents were also asked about the difficulty of the tasks solved by the BI system. The assessments appear in Error! Reference source not found.. Table 6 Univariate statistics on ‘Task difficulty’. Statement Min Max Mean SD BI makes it possible to complete complicated tasks. 1 5 3.12 0.984 The tasks I complete in BI require specialised knowledge. 1 5 3.05 1.127 The tasks I solve in BI are ones I have never faced before. 1 5 2.55 1.207 The assessments of the two first statements in Table 6 signal a neutral attitude. The latter statement, concerning the novelty of the tasks, demonstrates that the users, to some extent, consider BI tasks to be routine. Despite their ratings of the statements, the interviews reveal nuances of task difficulty. Thus, the interviews demonstrate examples of both routine and more complex tasks. To illustrate routine tasks, consider: “Well, if I have to do a monthly follow-up, then I need to define and follow up on every cost centre and see the transactions, if they are okay. That is like a routine task” (2017). “That is when I make a list of the patients we had for the last five years with a specific diagnosis. Super easy task, because the template was developed for that purpose. Some BI people have been thinking big thoughts, and there are very good headings for what you should go and look for in the system, so it is just a matter of going in and typing your filters” (2017). 51 However, the interviews also reveal examples of more complex tasks. For instance: “We had some where we should combine the kind of medicine they got, which is a standard extract in BI, with how long they were hospitalised. So, they should have had both a certain kind of medicine and be hospitalised for more than five days, for instance” (2017). Here, the complexity consists of combining different data types. Another kind of complexity is when the underlying data models are complex. For example: “Yes, you need to know your data and which… You could believe that you have the right data and then there is really something you didn’t take into account. I think I have tried that quite often, at least in the first couple of years I was working with this. That you think that you had everything under consideration and then there is some kind of twist of it” (2017). The last cluster of statements, regarding the users’ tasks, concerns the specificity of these tasks. The assessments appear in Table 7. The respondents’ assessments are average when rating to what extent the tasks are defined before they start solving them. There is a general agreement that the tasks can be solved in different ways. Again, the table indicates some extent of the routine tasks in the low rating of the repeatability of the tasks in the last statement. Table 7 Univariate statistics on ‘Task specificity’. Statement Min Max Mean SD My tasks are always defined before I complete them in BI. 1 5 3.03 1.061 The tasks I complete in BI can be done in more than one way. 1 5 3.29 0.935 Normally, I do not complete the same kinds of tasks in BI. 1 5 2.03 1.117 4.3 Users’ assessments of BI success In addition to the respondents’ background characteristics and the characteristics of their BI tasks, the survey also considered system and information quality as independent variables that influence the success of the BI system. The assessments of system quality appear in Table 8. In that table, all statements have mean ratings below 3, meaning that the users find the system difficult to learn, use and understand. The challenges are expressed in the interviews: “It requires quite a lot to learn how to use BI” (2017). “I would say that, about using the front end part of it, if you haven’t used it a lot, then it can be quite difficult to find out how to present it” (2017). Table 8 Univariate statistics on ‘System quality’ Statement Min Max Mean SD BI is easy to learn. 1 5 2.62 1.098 BI is easy to use. 1 5 2.74 1.094 The information in BI is easy to understand. 1 5 2.89 1.009 When the users have difficulties using the BI system, they report two strategies for the appropriation of the technology (Dourish, 2003). One is asking a colleague for help, which is considered an example of the employee aiming to adopt the technology. The other exemplifies adaptation. Here, the users import the data into Excel: “At times, I import it into Excel. I might as well admit it: I love Excel, including the graphical part. I like working with that” (2017). Table 9 Univariate statistics on ‘Information quality’. Statement Min Max Mean SD Data are displayed in a consistent format in BI. 1 5 3.11 0.948 The data in BI have high validity. 1 5 3.20 0.955 Other employees in the organisation also think the data in BI have a high degree of validity. 1 5 3.04 0.871 Information quality is another aspect that influences the users’ assessment of BI success. Three statements are included in the construct. The assessments appear in Table 9. Overall, the users have a neutral assessment of the three statements with a mean slightly above 3. 52 Thus, the users believe the consistency of the data to be reasonable. The data validity is rated slightly higher, while the users’ impressions of other employees’ impressions receive the lowest, but also most neutral, assessment. Regarding users’ satisfaction with the BI system (see Table 10), they do not think that the system’s functions and capabilities are as expected (rated at a mean of 2.82). However, they would still recommend the system to colleagues (rated at a mean of 3.21). The overall rating of satisfaction has a mean of 3.07. Table 10 Univariate statistics on ‘User satisfaction’. Statement Min Max Mean SD BI has all the functions and capabilities I expect it to have. 1 5 2.82 1.067 If a colleague asked, I would recommend BI. 1 5 3.21 1.161 Overall, how satisfied are you with BI? 1 5 3.07 1.014 The interviews revealed some of the issues the users experience with the system. In some cases, the users prefer to report in Excel. For instance: “It is not like it is working in the same way as a spread sheet with formulas and the like. It is a little more complicated and heavy to work with” (2017). The users’ individual impact is lower, when asked if they can make reports in BI effectively (mean of 2.98) and quickly (mean of 2.73). Completing the reports in BI is rated higher (mean of 3.04), suggesting that, although it may not be effective or fast, the users do finish their reports in the system (see Table 11). Table 11 Univariate statistics on ‘Individual impact’. Statement Min Max Mean SD I can effectively make my reports using BI. 1 5 2.98 1.105 I can complete my reports quickly using BI. 1 5 2.73 1.240 I can complete my reports using BI. 1 5 3.04 1.111 5. DISCUSSION The data analysis has shown that the majority of respondents and active users of BI are employees, and not managers as found in other studies. To most respondents, BI was not playing a dominant role in their work life, which may also explain their assessment of their own experience as being limited. However, the users handle routine tasks and more difficult tasks in the system. The most important task handled in the BI system was data extraction and more specifically filtering data and merging them into tables. The most important use of BI is internally in the organisations. The users do not think it is very easy to learn how to use the system, but they do experience consistency and validity of the data in the system, and they would recommend it to colleagues. The results of the study can be used to indicate how implementation can be approached to take into account the strengths and challenges users experience in using BI as a part of their work practice. The results demonstrate that the users still can experience challenges in using the system, although the system has been implemented for some time in all three case organisations. This paper used DeLone & McLean (1992) for guiding the data collection. That enables comparison across diverse organisations for a general picture of BI use and users in the public domain. However, if the aim is a more detailed understanding of the BI tasks and related use in subdomains within this domain, more task-oriented theories as presented in the theory section could generate a more detailed understanding of task characteristics and the system use generated on that basis. 6. CONCLUSION This paper has provided a picture of the characteristics of BI users and their tasks carried out by means of a BI system in different parts of the public sector. The analysis has not investigated differences between domains, but merely presents a cross sector perspective. Future research should aim to investigate further the differences between sub domains, the types of tasks generated, and the system success found as a consequence. This could provide useful inputs for the implementation of BI systems in the public sector. 53 7. REFERENCES Brynjolfsson, E., Hitt, L. M., & Kim, H. H. (2011). Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? SSRN Electronic Journal. https://doi.org/10.2139/ssrn.1819486 Byström Katriina, & Hansen Preben. (2005). Conceptual framework for tasks in information studies. Journal of the American Society for Information Science and Technology, 56(10), 1050–1061. https://doi.org/10.1002/asi.20197 Chen, H., Chiang, R. H., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4), 1165–1188. Choudrie, J. (2005). Investigating the Research Approaches for Examining Technology Adoption Issues. Journal of Research Practice, 1(1), D1. Crystal, A., & Ellington, B. (2004). Task analysis and human-computer interaction: approaches, techniques, and levels of analysis. AMCIS 2004 Proceedings, 391. DeLone, W. H., & McLean, E. R. (1992). Information Systems Success: The Quest for the Dependent Variable. Information Systems Research, 3(1), 60–95. https://doi.org/10.1287/isre.3.1.60 Dishaw, M. T., & Strong, D. M. (2003). The effect of task and tool experience on maintenance CASE tool usage. Information Resources Management Journal; Hershey, 16(3), 1–16. Dörfler, A. (2003). Business process modelling and help systems as part of KM in e- government (pp. 297–303). Presented at the IFIP International Working Conference on Knowledge Management in Electronic Government, Springer. Dourish, P. (2003). The Appropriation of Interactive Technologies: Some Lessons from Placeless Documents. Computer Supported Cooperative Work: The Journal of Collaborative Computing, 12(4), 465–490. Driscoll, D. L., Appiah-Yeboah, A., Salib, P., & Rupert, D. J. (2007). Merging qualitative and quantitative data in mixed methods research: How to and why not. Ecological and Environmental Anthropology (University of Georgia), 18. García, J. M. V., & Pinzón, B. H. D. (2017). Key success factors to business intelligence solution implementation. Journal of Intelligence Studies in Business, 7(1). Retrieved from https://ojs.hh.se/index.php/JISIB/article/view/ 215 Gartner Survey of More Than 2,500 CIOs Charts the Rise of the Digital Ecosystem. (n.d.). Retrieved March 27, 2017, from http://www.gartner.com/newsroom/id/348111 7 Iivari, J. (2005). An Empirical Test of the DeLone-McLean Model of Information System Success. SIGMIS Database, 36(2), 8–27. https://doi.org/10.1145/1066149.1066152 Ingwersen, P., & Järvelin, K. (2005). The Turn: Integration of Information Seeking and Retrieval in Context. Springer Netherlands. Retrieved from //www.springer.com/la/book/9781402038501 Jürgensen, J. (2012). The Effects of Digitalization of SU on the Work of SU Workers. CBS, Copenhagen. Kraemer, K. L., & Dedrick, J. (1997). Computing and public organizations. Journal of Public Administration Research and Theory, 7(1), 89–112. Li, Y., & Belkin, N. J. (2008). A faceted approach to conceptualizing tasks in information seeking. Information Processing & Management, 44(6), 1822–1837. https://doi.org/10.1016/j.ipm.2008.07.005 Marshall, T. E., Byrd, T. A., Gardiner, L. R., & Rainer, R. K. (2000). Technology Acceptance and Performance: An Investigation into Requisite Knowledge. Information Resources Management Journal, 13(3), 33–45. Negash, S., & Gray, P. (2008). Business intelligence. In Business intelligence: Handbook on decision support systems 2. Berlin: Springer. Retrieved from https://doi- org.zorac.aub.aau.dk/10.1007/978-3-540- 48716-6_9 Nielsen, J. (1994). Estimating the number of subjects needed for a thinking aloud test. International Journal of Human-Computer Studies, 41(3), 385–387. Petter, S., DeLone, W., & McLean, E. R. (2013). Information Systems Success: The Quest for the Independent Variables. Journal of Management Information Systems, 29(4), 7– 62. https://doi.org/10.2753/MIS0742- 1222290401 54 Rai, A., Lang, S. S., & Welker, R. B. (2002). Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis. Information Systems Research, 13(1), 50–69. Removed, A. (n.d.). Article removed. Salmasi, M. K., Talebpour, A., & Homayounvala, E. (2016). Identification and classification of organizational level competencies for BI success. Journal of Intelligence Studies in Business, 6(2), 17–33. Seddon, P. B., Staples, S., Patnayakuni, R., & Bowtell, M. (1999). Dimensions of information systems success. Communications of the AIS, 2(3es), 5. Snellen, I. (2002). Electronic governance: implications for citizens, politicians and public servants. International Review of Administrative Sciences, 68(2), 183–198. Stanton, N. A. (2006). Hierarchical task analysis: Developments, applications, and extensions. Applied Ergonomics, 37(1), 55–79. https://doi.org/10.1016/j.apergo.2005.06.003 Thompson, R. L., Higgins, C. A., & Howell, J. M. (1994). Influence of Experience on Personal Computer Utilization: Testing a Conceptual Model. Journal of Management Information Systems, 11(1), 167–187. https://doi.org/10.1080/07421222.1994.115180 35 Wixom, B., & Watson, H. (2010). The BI-Based Organization: International Journal of Business Intelligence Research, 1(1), 13–28. https://doi.org/10.4018/jbir.2010071702 Worley, J. H., Chatha, K. A., Weston, R. H., Aguirre, O., & Grabot, B. (2005). Implementation and optimisation of ERP systems: A better integration of processes, roles, knowledge and user competencies. Computers in Industry, 56(6), 620–638. https://doi.org/10.1016/j.compind.2005.03.006