Key Success Factors in Business Intelligence 

Szymon Adamala * and Linus Cidrin ** 

* Business Intelligence and data warehousing consultant, Poland, 

szymon.adamala@gmail.com 

 

** Management consultant, Sweden, 

linus.cidrin@hotmail.com 

 

Received 25 May 2011; received in revised form 3 June 2011; accepted 25 November2011 
 

ABSTRACT: Business Intelligence can bring critical capabilities to an organization, but the implementation of 

such capabilities is often plagued with problems. Why is it that certain projects fail, while others succeed? The 

aim of this article is to identify the factors that are present in successful Business Intelligence projects and to 

organize them into a framework of critical success factors. A survey was conducted during the spring of 2011 to 

collect primary data on Business Intelligence projects. Findings confirm that Business Intelligence projects are 

wrestling with both technological and non-technological problems, but the non-technological problems are 

found to be harder to solve as well as more time consuming than their counterparts. The study also shows that 

critical success factors for Business Intelligence projects are different from success factors for Information 

Systems projects in general. Business Intelligences projects have critical success factors that are unique to the 

subject matter. Major differences can be found primarily among non-technological factors, such as the presence 

of a specific business need and a clear vision to guide the project. Success depends on types of project funding, 

the business value provided by each iteration in the project and the alignment of the project to a strategic vision 

for Business Intelligence at large. Furthermore, the study provides a framework for critical success factors that, 

explains sixty-one percent of variability of success for projects. Areas which should be given special attention 

include making sure that the Business Intelligence solution is built with the end users in mind, that the Business 

Intelligence solution is closely tied to the company’s strategic vision and that the project is properly scoped and 

prioritized to concentrate on the best opportunities first. 

 

Keywords: Business Intelligence, Data Warehouse, Critical Success Factors, Enterprise Data Warehouse, 

Success Factors Framework, project risk management 

 

1. Introduction 

 

This article originated from a Master’s Thesis that 
the authors wrote as part of their MBA studies at 

Blekinge Institute of Technology, supervised by 

Dr. Klaus Solberg Søilen. 

The aim of the article is to develop the tools 

necessary to analyze, predict and manage the  

 

success of Business Intelligence, data warehousing 

and competitive intelligence initiatives in 

contemporary organizations. “There is no reason 
that each organization, as it begins and continues to 

develop data warehouse projects, must wrestle with 

many of the very difficult situations that have 

confounded other organizations. The same 

impossible situations continue to raise their ugly 

heads, often with surprisingly little relation to the 

Available for free online at https://ojs.hh.se/ 
Journal of Intelligence Studies in Business 1 (2011) 107-127 

mailto:kss@bth.se
mailto:philipbaback_orbsix@msn.com
https://ojs.hh.se/


108 
 

industry, the size of the organization, or the 

organizational structure” (Adelman, 2003). If 
there is indeed little correlation between success 

and the environmental factors of those initiatives, 

the cause of success must be related to something 

else. It is worth investigating whether the difficulty, 

complexity of initiatives and the high probability of 

failure are related to some factors inherent to 

Business Intelligence itself.  

In fact, most of the Business Intelligence and 

data warehouse projects fail and the failure rate is 

estimated to be between 50-80% (Beal, 2005; 

Meehan, 2011; Laskowski, 2001; Legodi & Barry, 

2010). Why do so many of these projects fail? “
The warehouse is used differently than other data 

and systems built by information systems” 
(Inmon, 2002). Therefore, the available knowledge 

on how to manage ordinary projects in the 

information systems domain should not be 

expected to be readily applicable to the area of 

Business Intelligence. 

Our initial information search revealed that 

Business Intelligence has been covered in a large 

number of academic articles and books. However, 

Business Intelligence success is not widely treated. 

Few articles attempt to analyze what the factors are 

which influencing BI. Some of them have 

developed frameworks, but there is no proven 

framework that can be relied upon for this 

particular problem.  

The success of BI initiatives undertaken by 

companies depends on various factors. Because BI 

implementation depends on the successful use of IT 

resources, some of those factors are undoubtedly 

technological. However, Information Technology 

is merely a tool in implementing a BI solution. BI 

success depends chiefly on organizational and 

process factors that business administration focuses 

on. Despite differences in BI systems between 

various industries, those factors should be 

identified and classified. The existence of a link 

between individual factors or their combinations 

and the success of BI initiatives should be verified. 

Moreover, success factors could be classified and a 

method of measurement could be devised for each 

individual factor. As the management thinker Peter 

Drucker once said, “what gets measured, gets 

managed”. Currently, BI initiatives have a 
relatively low success ratio. The successful ones 

achieve ROI in excess of 400% (according to some 

vendors). This figure is even more impressive 

when taking into account the (traditionally capital) 

investments in the infrastructure and equally high 

(traditionally operational) expenses on 

development projects, operations, consulting 

services etc. An ability to assess the chances of 

success prior to engaging in the initiative would 

have two practical benefits: 

 

1. Preventing some of the initiatives from 

commencing if they score low in the 

framework of success factors or allowing 

them to focus on critical issues that otherwise 

would have caused a failure - as a result, the 

success ratio of all BI initiatives improves. 

2. Facilitating management of the investment 

in Business Intelligence and lowering the 

entry barriers (given the high level of 

investment, 50% probability of failure might 

be too high for smaller companies if the loss 

resulting from failure push the company into 

bankruptcy). 

 

The article focus on the following hypotheses: 

 

Hypothesis 1. It is possible to identify critical 

success factors (CSFs) of BI initiatives and 

they are different from the CSFs of 

information systems (IS) in general. 

Hypothesis 2. Non-technical and non 

technological CSFs play a dominant role in BI 

initiatives’ success. 

Hypothesis 3. Technology plays a secondary 

role and is less critical for BI initiatives’ 

success. 

Hypothesis 4. It is possible to develop 

objective measures for each of CSFs of BI 

initiatives. 

 

If the abovementioned hypotheses can be proved, 

CSFs of BI initiatives can create a framework that 

can be used to predict the success in early stages of 

BI initiatives.  

It should be noted that, even though this study 

is primarily focused on vendors in Poland, the 

conclusions from this study may have relevance 

and be applicable to other countries as well, 

especially since many of the companies studied 

were global enterprises and other aspects of 

projects and IT are global: Outsourcing, offshoring, 

nearshoring and other forms of international 

operations. the differences in survey results 

between Poland and other countries were not 

significant and drawing conclusions from the 

sample we gathered, we claim that our findings are 

also applicable to Business Intelligence projects in 

larger enterprises worldwide. The reason for 

choosing Poland as the primary country is that one 

of the authors has been working there for many 

years and had good access to empirical data. 

There is some ambiguity around the use of the 

terms data warehouse and Business Intelligence 

among practitioners. Historically, data warehousing 

meant “the overall process of providing 

information to support business decision making” 
(Kimball, Ross, Thornthwaite, Mondy & Becker, 

2008, p. 10). Following these authors, “delivering 
the entire end-to-end solution, from the source 

extracts to the queries and applications that the 



109 
 

business users interact with” (Kimball et 
al.,,2008, p. 10) has always been a fundamental 

principle. However, the term Business Intelligence 

coined in the 1990s meant only reporting and 

analysis of data stored in the warehouse, effectively 

separating the two terms. Even now, there is no 

agreement in the industry – some “refer to data 
warehousing as the overall umbrella term, with the 

data warehouse databases and BI layers as subset 

deliverables within that context” (Kimball, et al.,  

2008, p. 10), while others “refer to business 
intelligence as the overarching term, with the data 

warehouse relegated to describe the central data 

store foundation of the overall business intelligence 

environment” (Kimball et al., 2008, p. 10).   
Because the aim of this article is not to define 

the term of Business Intelligence, but rather to 

investigate its success factors, both conventions are 

accommodated. The literature referring both to data 

warehouse systems/projects and Business 

Intelligence systems have been reviewed. There is 

no difference in their relevance to our study, 

especially when taken into account the industry 

practice that even “though some would agree that 
you can theoretically deliver BI without a data 

warehouse, and vice versa, that is ill-advised /…/. 
Linking the two together in the DW/BI acronym 

further reinforces their dependency” (Kimball et 
al., 2008, p. 10). Despite this argumentation, 

Business Intelligence is used in this article as an 

overall term.  

The research is concentrated mainly on the 

factors that lead to the success of BI initiatives, 

cause its success (or are able to prevent it), or are 

otherwise correlated with the success. However, 

little attention is paid to the definition of the 

success itself or its measurement. According to 

Delone and McLean (1992; 2003), success in 

business entails net benefits. To properly define 

success, those net benefits need to be measured. 

The authors’ goal of obtaining objective 
measurement criteria would call for a method to 

compute the NPV of a BI initiative. A similar 

subject (without the emphasis on BI, but rather for 

software in general) has recently been a topic of a 

PhD thesis at Blekinge Institute of Technology 

(Numminen, 2010). Clearly, calculating the NPV 

of BI initiatives will be out of scope for this article. 

Therefore, the success itself will not be defined as 

clearly as the authors should want to at first. This 

would instead be a suggestion for further research 

in this area. 

Several of the models proposed in the 

literature (Delone & McLean, 1992; 2003; Hwang 

& Xu, 2008; Wixom & Watson, 2001) analyze the 

interdependency between independent variables. 

Furthermore, none of the abovementioned authors 

propose a direct correlation with success for those 

variables that were used to explain variability of 

other independent variables. The goal of this study 

is to enumerate all the CSFs of BI projects, not to 

analyze how they depend on each other. For each 

of the independent variables analyzed, a direct 

dependency between success and that variable is 

analyzed.       

 

2. Literature review 

 

Dr Ralph Kimball is credited with developing 

arguably the most successful data warehouse 

architecture, the dimensional modeling. The 

success criteria as described by Kimball are named 

“readiness factors”. The author suggests giving 
up the initiative, if some of the criteria are missing 

or insufficient. “Readiness shortfalls represent 
project risks; the shortfalls will not correct 

themselves over time. It is far better to pull the plug 

on the project before significant investments have 

been made than it is to continue marching down a 

path filled with hazards and obstacles” (Kimball 
R., Ross, Thornthwaite, Mondy, & Becker, 2008, p. 

16). 

 Let us examine the existing frameworks that 

might be applicable for explaining Business 

Intelligence success.  

 

2.1 Information Systems Success 

The most obvious first choice when trying to 

discover BI success factors is to look at 

information systems (IS) in general. There exists an 

excellent framework proposed by Delone and 

McLean (Delone & McLean, 1992; 2003) that has 

been widely cited, revisited, criticized, validated or 

extended in hundreds of articles (Delone & 

McLean, 2003). The original framework proposed 

by Delone and McLean (1992) has been 

reexamined in the light of that subsequent research, 

resulting in a slightly updated version (Delone & 

McLean, 2003) that will be discussed here. The 

proposed framework defines IS success in terms of 

system use, user satisfaction and net benefits 

whereas the factors leading to the success 

encompass information quality, system quality and 

service quality. The interdependences between 

those variables are depicted in Figure 1. 



110 
 

 
Figure 1: Delone&McLean model for IS success (Delone & McLean, 2003) 

Our study did not measure system quality and 

service quality at all, as they are inapplicable to 

Business Intelligence. As Solberg Søilen and 

Hasslinger (2009) found, vendors of BI tools do not 

differentiate their products and tools (other than 

adopting different definitions of Business 

Intelligence). Therefore, system quality should not 

be a factor in the Business Intelligence domain. 

Because Business Intelligence systems are built to 

present data and facilitate its analysis, data quality 

is probably the most important factor. From the “

second tier” variables, intention to use and user 
satisfaction should play a small role in Business 

Intelligence. If BI is tied to strategic vision for the 

entire company, users are obliged to use the 

solution and not their own data sources. Therefore, 

use alone as a variable plays a role.  

One major drawback with this framework that 

we identified is the concentration on technology. 

The findings of our research dictate that the success 

of BI initiatives relies on numerous factors that are 

within the management domain rather than on 

technological factors. Therefore, although the 

Delone and McLean model of IS success has been 

applicable to many specific information systems, 

even those that have been unforeseen when the 

framework was originally constructed (Delone & 

McLean, 2003), we think it cannot be applied to 

Business Intelligence systems without due changes.  

Delone and McLean’s framework not only 

does not propose any specific measurement 

methods, but also avoids specific variable 

definitions, leaving them to be concretized at the 

time of framework application, depending on the 

specific context (Delone & McLean, 2003).  

 

2.2 Critical Success Factors for BI Systems 

 

“The implementation of a BI system is not a 

conventional application-based IT project (such as 

an operational or transactional system), which has 

been the focus of many CSF studies” (Yeoh & 

Koronios, 2010). Having noted that, the authors 

proceeded to propose a framework that included 

some of the critical success factors or success 

variables from Delone & McLean (1992) as a part. 

System quality, information quality and system use 

were grouped together and labeled “infrastructure 

performance”. As an equal factors group, process 

performance was proposed, encompassing classical 

project management variables like budgets and 

time schedules. The authors emphasized a different 

set of factors, divided into three broad categories – 

organization (vision and business case related 

factors, management and championship related 

factors), process (team related factors, project 

management and methodology related factors, 

change management related factors) and 

technology (data related factors, infrastructure 

related factors). All those factors cause business  



111 
 

  

Figure 2: Yeoh & Koronios model of success in BI (Yeoh & Koronios, 2010) 

 

orientation, which in turn, together with before 

mentioned infrastructure and process performance 

factors lead to implementation success and, 

subsequently, to perceived business benefit. This 

framework is illustrated in Figure 2.  

 

 

 

 

 

A summary of success factors across all the 

dimensions with the highlight of the most 

important points: 

 

Organizational dimension 

 Committed management support and 
sponsorship 

 Clear vision and well-established business 
case 

Process dimension 

 Business-centric championship and 
balanced team composition 

 Business-driven and iterative development 
approach 

 User-oriented change management 
Technological dimension 

 Business-driven, scalable and flexible 
technical framework 

 Sustainable data quality and integrity 
 

This framework is considerably better suited for 

Business Intelligence systems than the one 

presented by Delone and McLean (2003). 

However, it has a few drawbacks. First, there are 

no specific measurement criteria proposed. Since 

several of the variables are defined in a way so that 

very different measurement criteria can be defined 

or it would be hard to devise measures other than 

the Likert scale, the framework would be 

impractical to apply and the results might depend 

on the subjective opinions of those that provide or 

calculate variable’s values, resulting in the false 
prediction of BI initiative success. Another 

drawback is an artificial reliance on the Delone and 

McLean (2003) model of IS success. Some 

infrastructure performance factors (system quality, 

information quality) belong under the technology 

category (infrastructure related factors or data 

related factors) and should not be repeated at all or 

the causal dependency should be included in the 

framework. 

Further criticism of this article (Yeoh & 

Koronios, 2010) can be based on the way inclusion 

of technology related factors was performed. “
Business-driven, scalable and flexible technical 

framework” and particular variables that fall into 
this category are too close to the way the vendor 

who delivered the solution gives advice on his or 

her product. Also, the fact that this article (Yeoh & 

Koronios, 2010) was based on a single project 



112 
 

plays down the objectivity of its findings. As our 

survey later revealed, technical issues are of 

secondary importance. 

 

2.3 Impossible Data Warehouse Situations. 

 

Why are the success criteria so important? As we 

have already mentioned in the introduction, some 

data warehouse implementations fail. It is possible 

that many of these ‘failures’ would be 
considered successes (or at least to be positioned 

someplace between failure and success). The 

criteria for success are often determined afterwards. 

This is a dangerous practice since those involved 

with the project don’t really know their targets 
and will make poor decisions about where to put 

their energies and resources (Inmon, 2002). 

According to Adelman (2003), the example 

measures of success include the following 

elements: 

 

 The data warehouse usage. 

 Usefulness of the data warehouse, including 
end user satisfaction. 

 Acceptable performance as perceived and 
experienced by end users. 

 Acceptable ROI - benefits justifying the 
costs. 

 Relevance of data warehouse application. 

 Proper leverage of a business opportunity 
(i.e. establishing competitive advantage 

thank to the application of data warehouse). 

 Timely answers to business questions. 

 Higher quality / cleanliness of data (than in 
the source systems). 

 Initiation of changes in the business 
processes that can be improved as a result of 

the use of data warehouse (Adelman, 2003, 

p. 5 & 40). 

 

Douglas Hackney emphasized the critical 

importance of establishing success measures – “

The ‘build it and they will come’ approach (…) 

does not work in data warehousing” (Adelman, 
2003, p. 41). He further suggests starting with 

business “pain” with the available data. The 

words “life-threatening” seem to accurately 
describe the problem that need to be chosen to 

solve (Inmon, 2002, p. 41). An example of a 

business “pain” might be the “loss of revenue 

or excessive operating costs” (Inmon, 2002, p. 
42). 

The main theme of the book of Adelman 

(2003) is not the success factors of Business 

Intelligence initiatives. Instead, the author 

concentrates on various difficult situations and 

proposes methods to overcome those adversities. 

Although he does not name those factors as 

influencing the success itself, some of them should 

at least be evaluated for possible inclusion in the 

final framework of this study. According to the 

author, all difficulties can be organized in one of 

the following categories (the first seven represent 

management issues, the rest is concerned with the 

technical side): 

 

1. Management issues 
2. Changing requirements and objectives 
3. Justification and budget 
4. Organization and staffing 
5. User issues 
6. Team issues 
7. Project planning and scheduling 
8. Data warehouse standards 
9. Tools and vendors 
10. Security 
11. Data quality 
12. Integration 
13. Data warehouse architecture 
14. Performance 
 

From the factors identified in the book of Adelman 

(2003), we found a few that were less important 

than others, while some were not considered at all 

in our survey as separate variables. For example, 

architecture, performance, security, tools and 

standards were considered as one broad variable 

technical issue in our research and were deemed of 

secondary importance. Data quality, on the other 

hand, was measured in our survey separately for 

source systems and target solution, both being 

found to have importance.  

 Adelman ( 2003) devote at least equal 

attention to technical issues in general as to all non-

technical issues, the latter being mentioned in less 

detailed discussions. Broad and general categories 

of “user issues”, “team issues”, “

organization and staffing”  etc.  make detailed 
comparison with our study difficult. Also, a 

different terminology plays a role here. What the 

author considers measures, our study call variables. 

For example, “data warehouse usage” was a 
measure in the book (Adelman, 2003) while as we 

propose three distinct measurement methods, 

namely number of queries per period, number of 

logons per period and number of users per period.  

 

2.4 Competing Value Model 

 

Since different stakeholders will look at different 

criteria to determine if the initiative was a 

successful one, different perspectives may need to 

be taken into account. This issue of potentially 

conflicting criteria has been discussed by Walton 

and Dawson in their Competing Value Model 

which organizes criteria in different dimensions 

(Walton & Dawson, 2001). This could be present 

in BI initiatives as well, where different 



113 
 

stakeholders can have different non-overlapping 

criteria for success; for example a system integrator 

can have different objectives for engaging in a BI 

initiative, different from the end user organization 

(the customer). 

This study did not attempt to define BI 

initiative success, but considered using this kind of 

classification of criteria for different dimensions, 

such as technical and non-technical. The definition 

of success of BI initiatives is not part of this study. 

One consequence of the findings of the article by 

Walton & Dawson (2001) is that the development 

of a framework allows for the interest of different 

stakeholders to be organized. This problem is left 

for future studies. 

 

2.5 A Critical Success Factor Framework for 

Implementing Business Intelligence Systems 

 

Qualitative studies have been used in the area of 

critical success factors in Business Intelligence 

many times. One of these is a Critical Success 

Factor Framework for Implementing Business 

Intelligence Systems (Yeoh, Gao, & Koronios, 

2007) where the authors performed a Delphi study 

covering 15 BI systems experts to define a critical 

success factor framework for implementing 

business intelligence systems.  The authors propose 

a framework that is organized into seven 

dimensions covering 22 factors. The seven 

dimensions of this article (Yeoh, Gao, & Koronios, 

2007) are: 

 

 Commitment management support and 
championship 

 User-oriented change management 

 Business vision 

 Project planning 

 Team skills and composition 

 Infrastructure-related dimensions 

 Data related issues 
 

The Delphi panel rates the factors in those 

dimensions and a mean is calculated for each 

dimension. The above list of dimensions is ordered 

in a falling mean ranking, starting with the 

dimension with the highest rating. We see that 

more technical dimensions can be found at the very 

bottom of the list, while as non-technical 

dimensions are found higher up in the list. 

The paper by Yeoh et al. (2007) demonstrates, 

in line with the result from this study, that 

management championship and the presence of a 

clear vision for Business Intelligence is important 

when executing a Business Intelligence initiative. 

While the result from this study seems to indicate 

that the technical factors are more important than 

the non-technical factors, the authors of the paper 

include two “technical” dimensions out of seven. 
It should be said that the ratings of the two 

technical dimensions are the lowest out of the 

seven, which is well in line with the findings of this 

study. 

 

2.6 Risk Management in Enterprise Data 

Warehouse Projects in South Africa 

 

The paper starts with the claim that data 

warehousing implementation projects have high 

estimated failure rates, up to about 50% (Legodi & 

Barry, 2010). The objective of the paper and the 

study was to investigate the main areas of risk for 

these projects, and a Delphi method was used for 

the data collection. The study was performed in 

South Africa. Results from the study are in the 

form of the main problems and the success factors 

for this type of project. The identified factors that 

have the greatest impact on success from the study 

(Legodi & Barry, 2010) are (listed in priority 

order): 

 

1. Scope Creep 
2. Uncontrolled Finances 
3. Poor Communication 
4. Stake Holder Non-Involvement 
5. Skills Shortage 
6. Unavailability of Tools and Technology 
7. Uncontrolled Quality of Deliverables 
8. Poor, Wrong or No Leader 
9. Technical Difficulties 
10. Legal Difficulties 

 

We note that the first technical difficulties can be 

found in the ninth item. 

(Legodi & Barry, 2010) show, like the results 

from this study, that non-technical factors affect the 

success of a project the most. Out of the ten factors 

Legodi and Barry (2010) identified, only one were 

of technical nature and placed as the ninth factor 

(technical difficulties). The fourth item, 

stakeholder non-involvement, can be comparable to 

the importance of vision and addressing a specific 

business need of the sponsor. While the paper 

surveyed a more narrowly defined geography 

(South Africa), the finding that non-technological 

factors dominate is shared between that paper and 

this study. 

 

2.7 An Exploratory Investigation of System Success 

Factors in Data Warehousing 

 

The variables used in a study by Shin (2003) are 

system throughput, ease of use, ability to locate 

data, access authorization, data quality (subdivided 

into 4 more detailed categories of 

recency/currency, level of detail, accuracy and 

consistency), information utility, user training and 

user satisfaction, with the latter is being used as 

dependent variable. The data was gathered from a 

single large US enterprise, based on a single 



114 
 

project, therefore even the author agrees that this 

study must be treated as a case study (Shin, 2003, 

p. 157). The authors found that 70% of end user 

satisfaction could be explained by the independent 

variables that were measured. 

Shin (2003) approaches data warehousing 

success in a different way than in the above study, 

treating end user satisfaction as a proxy for project 

success. This is also inconsistent with Delone and 

McLean approach (Delone & McLean, 1992; 2003) 

which Shin cited and claimed to be “the most 

influential model in conducting research on 

information systems success factors” (Shin, 2003, 

p. 144). The variables used are mainly different 

from the inspected ones, with only data quality 

being common with this study. Variables used by 

Shin (2003) describe predominantly technological 

factors. As in this study actual use is only one of 

the variables, the results are hardly comparable. 

Also, a case study approach concentrated on 

success within just one company makes this article 

(Shin, 2003) less valuable for creating a larger data 

warehousing or Business Intelligence success 

criteria framework.  

A Structural Model of Data Warehousing 

Success by Hwang & Xu (2008) analyzes several 

factors influencing data warehousing success. 

Success is not measured as a separate variable. 

Instead, following Delone and McLean (1992; 

2003), benefits and quality are used as proxy, with 

a distinction between individual and organizational 

benefits as well as system and information quality. 

The independent variables are; operational factor, 

technical factor, schedule factor and economic 

factor. The authors propose at least two measures 

for each variable, dependent and independent. The 

model (Figure 3) is able to explain between 27% 

(system quality) and 41% (organizational benefits) 

of the dependent variables variability.  

 

 

Figure 3: Hwang & Xu structural model of data warehousing success (Hwang & Xu, 2008) 

The most prominent difference in approaches is the 

definition of what a variable is and what a measure 

is. Many measures from this article (Hwang & Xu, 

2008) are variables in our study, for example 

business benefits definition, business benefits 

measurability or scoping of a project. The aim for 

objective measurement methods were not one of 

the goals of Hwang & Xu (2008) and it is unclear 

how measures proposed in their work should be 

performed in practice. The authors were overly 

reliant on the model of IS success (Delone & 

McLean, 1992; 2003), obtaining little explanatory 

power for their model. From the way the model 

was constructed, it is also hard to compare whether 

technical or non-technical factors play major role in 

data warehousing success.  

An Empirical Investigation of the Factors Affecting 

Data Warehousing Success (Wixom & Watson, 

2001) examined the factors influencing data 

warehousing success. Authors verified several 

hypotheses. Findings can be summarized as 

follows:  

 

Hypothesis 1a. A high level of data quality 

will be associated with a high level of 

perceived net benefits (supported).  

Hypothesis 1b. A high level of system quality 

will be associated with a high level of 

perceived net benefits (supported).  

Hypothesis 2a. A high level of organizational 

implementation success is associated with a 

high level of data quality (not supported). 



115 
 

Hypothesis 2b. A high level of organizational 

implementation success is associated with a 

high level of system quality (supported).  

Hypothesis 3a. A high level of project 

implementation success is associated with a 

high level of data quality (not supported). 

Hypothesis 3b. A high level of project 

implementation success is associated with a 

high level of system quality (supported).  

Hypothesis 4a. A high level of technical 

implementation success is associated with a 

high level of data quality (not supported). 

Hypothesis 4b. A high level of technical 

implementation success is associated with a 

high level of system quality (not supported). 

Hypothesis 5. A high level of management 

support is associated with a high level of 

organizational implementation success 

(supported).   

Hypothesis 6a. A strong champion presence is 

associated with a high level of organizational 

implementation success (not supported). 

Hypothesis 6b. A strong champion presence is 

associated with a high level of project 

implementation success (not supported). 

Hypothesis 7a. A high level of resources is 

associated with a high level of organizational 

implementation success (supported).  

Hypothesis 7b. A high level of resources is 

associated with a high level of project 

implementation success (supported).  

Hypothesis 8a. A high level of user 

participation is associated with organizational 

implementation success (supported).  

Hypothesis 8b. A high level of user 

participation is associated with project 

implementation success (supported).  

Hypothesis 9a. A high level of team skills is 

associated with project implementation success 

(supported).  

Hypothesis 9b. A high level of team skills is 

associated with technical implementation 

success (not supported). 

Hypothesis 10. High-quality source systems 

are associated with technical implementation 

success (supported). 

Hypothesis 11. Better development technology 

is associated with technical implementation 

success (supported). 

 

A valuable part of in this article (Wixom & 

Watson, 2001) that even led to verifying the 

hypotheses proposed in this paper, was the 

statistical model using partial least squares 

regression. The resulting model can be seen in 

Figure 4. 

 

 

Figure 4: Wixom&Watson statistical model of DW success (Wixom & Watson, 2001) 

R
2
 values obtained by Wixom and Watson (2001), 

(Figure 4) ranging between 0.016 to 0.435, means 

that independent variables used in the model 

provide limited explanatory power for dependent 

variables. This makes it impossible to compare 

findings with our study. The model relies on the IS 

success model by Delone & McLean (1992) which 

defines system success using a structure of proxy 

variables – data and system quality, and benefits. 
Variables used covered both technical and non-

technical factors influencing data warehouse 

implementation success. The way the model was 



116 
 

constructed makes it impossible to verify whether 

technical or non-technical factors influence the 

success more strongly. The article (Wixom & 

Watson, 2001) does not attempt to propose specific 

measurement criteria for the variables used. 

 

2.8 Current Practices in Data Warehousing 

 

The study conducted in the article of Watson, 

Annino, Wixom, Avery, & Rutherford (2001) 

concentrated on some of the factors influencing 

data warehousing projects success. Survey 

respondents were asked to provide answers to 

questions about who sponsored the data warehouse, 

which organization unit was the driving force 

behind the initiative, about solution architecture 

and end users, about implementation costs, 

operational costs, solution approval process, after 

implementation assessment, realization of expected 

benefits (together with expectations). To describe 

success, two questions were used, one about ROI 

and the other about perceived success of 

implementation.  

Although the authors gathered enough data to 

analyze the dependency of individual factors with 

success or to build a statistical model of several 

variables, they chose not to do so. Instead, they 

analyzed individual questions, summarizing 

contemporary practices. Findings say that most 

data warehouse sponsors come from business units 

and no single architecture dominates all 

implementations (Watson, Annino, Wixom, Avery, 

& Rutherford, 2001). Because of the lack of a 

proposed framework, it is impossible to compare 

this article with our findings. Also, the article did 

not attempt to propose any objective measurement 

methods for variables included in the study. 

 

2.9 Measuring User Satisfaction with Data 

Warehouses: An Exploratory Study 

 

The study of Chen, Soliman, Mao, & Frolick, 

(2000) uses end user satisfaction as a proxy for 

success and examines factors influencing the 

aforementioned user satisfaction. Study results 

identify three factors comprising several questions 

each. Those factors are: support provided to end 

users, accuracy, format and preciseness of data, and 

fulfillment of end user needs. Authors did not 

propose a framework for success and did not 

attempt to build a statistical model. They also did 

not attempt to develop objective and universal 

measurement methods for the variables identified. 

Treatment of end user satisfaction as a proxy 

for project success and the goal to identify only the 

factors that influence this variable makes it difficult 

to compare the results with the findings. The 

authors (Chen, Soliman, Mao, & Frolick, 2000) 

claim to have relied on the model of IS success 

(Delone & McLean, 1992), but included only one 

of the “use” variables presented in that study. 
The study treated use as a whole and positioned it 

as one of the factors influencing success. It is 

impossible to tell whether technical or non-

technical factors are more important to project 

success following this article (Chen, Soliman, Mao, 

& Frolick, 2000) and no objective measurement 

methods are proposed. 

 

3. Empirical Data 

 

For primary research, surveys were used and 68 

fully completed surveys obtained. The results were 

analyzed using quantitative methods - simple 

statistics (mean, mode, median) performed on 

individual questions, correlation analysis of 

individual variables with the dependent variable of 

success and Partial Least Squares Regression (PLS-

R) used to build the target framework. Internet-

based e-surveys were used.  

Each single data point / measurement was a 

distinct Business Intelligence project. This implies 

that several measurements could have been 

obtained from a single vendor, customer or 

individual, provided that each of those 

measurements described distinct projects. Each 

survey contained optional questions about customer 

and vendor company names, project names, place 

where the project was performed as well as 

whether the survey participants were involved in a 

project on the customer or vendor side, and a 

precise project role of the participants. 

Initial secondary research revealed that some 

of independent variables chosen by authors are 

correlated. Because of this and the fact that we 

aimed at analyzing direct dependency of all 

independent variables with success, there was a 

very high probability of multicollinearity in the 

data. For this reason, a method of data analysis 

insensitive for multicollinearity had to be chosen. 

Also, because of high number of independent 

variables chosen based on phase 1 findings and 

relatively low number of observations obtained 

through the survey, an appropriate method had to 

be chosen. Both requirements combined were 

satisfied by Partial Least Squares regression, a 

method that “was designed to deal with (…) 
regression when data has small sample, missing 

values, or multicollinearity” (Pirouz, 2006).  
After completing the primary research, we 

propose our own framework for BI initiatives 

success, using prior research as a starting point. 

The framework was separated from the 

measurement methods to maintain the flexibility 

and applicability of the framework in various 

contexts. However, measurement methods were 

also proposed. Keeping those two separated will 

allow any future studies to propose different 

measurement for the variables. Also, a fact of 



117 
 

importance is that it is much easier to combine a 

theoretical framework with specific measurement 

methods if they are kept separated, than to split 

them if they are merged in a single framework. 

Therefore, we decided to separately propose a 

theoretical framework and a specific measurement 

method for each of the variables. 

 

4. Analysis and Implications 

 

As mentioned, this study excludes trying to define 

what BI project success looks like. Instead, to 

determine whether a project is successful or not, 

the following interpretation from the survey results 

have been made throughout the article. 

 

 Successful project/initiative - Project with 

answers “definitely yes” and “rather yes

” to the question Q19:1 -  “Was the 

project you are describing successful?” 

 Non-successful project/initiative - Project 

with answers “neutral”, “rather no” 

and “definitely no” to the question Q19:1 

- “Was the project you are describing 

successful?”. That is, non-successful 
means unsuccessful or neutral. 

 

The reason behind such a distinction is that this 

article aims for developing tools that managers 

could use to achieve success. Merely avoiding 

failure is not good enough. If the project failed to 

achieve success, it cannot be treated as a mild 

success or neutral. Instead, it should then be 

considered as a non-successful project. 

Results from the survey indicate that the 

funding of successful initiatives and non-successful 

ones differs. Successful initiatives are more likely 

to be funded in a phased approach, with separate 

budgets for each iteration, while the non-successful 

tend to be funded in a traditional way as other IT 

projects, i.e. one budget for the entire project.  

Iterative development is used by both 

successful and non-successful initiatives, but 

results from the survey show that for an 

overwhelming majority (89%) of the successful 

initiatives, each of the iterations provided business 

value by themselves. This can be compared to the 

non-successful initiatives that used an iterative 

approach, where every other initiative had 

iterations that provided value by themselves.  

A BI solution designed with end users in mind 

will naturally have a greater usage than BI 

solutions that don’t consider the end users. A 
whopping 96% of the successful initiatives in this 

study have users that actually use the Business 

Intelligence solution. The results for the non-

successful ones vary.  

The presence of an overarching, clear 

strategic vision for Business Intelligence within the 

organization is important for the success of an 

initiative. Results show that successful initiatives 

exist in an environment with a clear strategic vision 

to guide the initiative as compared to non-

successful ones. A total of 81% of the successful 

initiatives agrees to this compared to 46% of the 

non-successful. 

Not only is the existence of a strategic vision 

for Business Intelligence important, the business 

need that the initiative tries to solve should be 

aligned to that vision. Like the results for the 

existence of vision, the successful initiatives have 

business needs that tend to be more aligned with 

the strategic vision than for non-successful ones. 

73% of the successful initiatives agree to that. 

Corresponding figure for the non-successful 

initiatives is 39%.  

The survey shows that successful initiatives 

often concentrate on choosing the best 

opportunities for the project scope. 57% of the 

surveyed successful initiatives do this, compared to 

23% of the non-successful ones.  

Another parameter measured by the study is 

whether an expert assigned to a team or made 

available to a team, always was the best expert 

available. The results from the study reveal, 

somewhat surprisingly, that non-successful 

initiatives to a larger extent assigns the best expert 

available, with 85%. Corresponding number for the 

successful ones are 54%. 

Issues will occur during the duration of an 

initiative, which can be both technological and 

non-technological in nature. Both successful and 

non-successful initiatives do, to some extent, 

encounter non-technological issues, with non-

successful encountering them slightly more often 

according to this study. The technological issues 

are commonly found in both successful and non-

successful initiatives, but they are somewhat more 

common in the non-successful initiatives (77% 

compared to 63% reports presence of technological 

issues). For all initiatives in the study only slightly 

more than 20% found technology related problems 

to be the hardest. 

Consultants spend time on solving different 

kinds of problems, technological as well as non-

technological and for a majority of the initiatives a 

larger part of the time is spent on resolving non-

technological problems. This is especially true 

when looking at the non-successful initiatives, 

which show that almost 40% of the surveyed 

initiatives indicated that non-technological issues 

took the most time.  

 

4.1 A Statistical model of Business Intelligence 

initiatives’ success. 

 

The survey results were analyzed using the Partial 

Least Squares regression method. Two separate 

series of models were built, one using original, 



118 
 

untransformed data and the other, using the data 

transformed by centering on mean, dividing by a 

factor and applying basic mathematical functions. 

More details, and the rationale for performing those 

transformations, are provided later in the text. 

 The models built for original data analysis 

used 25 variables defined by answers to survey 

question nr 19. For the clarity of information 

presentation, variables were named after their 

respective survey questions – variable A19_1 
corresponded to question 19:1 answers, A19_2 

corresponded to question 19:2 answers and so on. 

Because question 19:1 asked users whether the 

project they described was successful, this variable 

was chosen as dependent for all the models, while 

every other variable was treated as independent. 

Unlike other linear regression methods, 

adding more variables does not necessarily imply 

an increase of model’s R
2
 when PLS is being 

used. The method cannot be used to eliminate 

insignificant variables, which is easily achieved in 

other linear regression methods. Therefore, the 

model that used all the available variables was not 

necessarily the best. Because the method cannot 

eliminate insignificant variables by setting the 

linear combination coefficients to 0 (or even close 

to 0), models that used only some of the variables 

were built.  

Choice of variables that explain success 

should be made based on the individual 

independent variables’ correlation with the 
dependent variable. After some experimentation, 5 

variables with correlation of success stronger than 

0.4 were chosen (absolute value of correlation 

coefficient greater than 0.4). 

 

Figure 5: PLS regression model with 5 significant variables 

As can be seen the model achieved a R
2 

of 0.559, 

with changes in independent variables explaining 

55.9% of changes in dependent variable. The 

variables that enabled these results were obtained 

as answers to the following questions: 

 

Question 19_3. Did the end users use the 

data warehouse / BI solution provided? 

Question 19_6. Were the business needs that 

project tried to solve aligned to the 

abovementioned strategic vision? 

Question 19_10. Did project scope 

definition concentrate on choosing the best 

opportunities ("low-hanging fruit")? 

Question 19_19. If technological issues 

were present, were all of them resolved? 

Question 19_20. Were non-technological 

issues encountered during the project? 

 

PLS is a linear regression method, but it is most 

likely that a dependency of success from the factors 

is not linear. The possibility of other dependencies 

had to be explored. While other, non-linear 

regressions could possibly be applied, a lack of any 

knowledge of the nature of the dependency would 

imply the need to try all of them, which was clearly 

beyond the scope of this study. Instead, linear 

regression could be applied to the transformed 

factors. For example, if linear regression could be 

done on squared factors, the outcome would be a 

case of square regression for the original factors; if 

various powers could be applied to these factors, 

the outcome would be polynomial regression and 

so on. 

It is always possible to choose a function that 

will fit the given finite, discrete data set. To avoid 

such a pitfall, the transformations had to be limited 

to basic mathematical functions – power (including 
square root), logarithm, exponential functions, 

reciprocal or simple composition of 

aforementioned functions. Similar selection 

criterion as for original data applied, but the 

transformation had to improve the data. The 

transformed data was considered better than 

original if for a given function f and variable x, f(x) 

had a stronger correlation with success than 

variable x itself.  



119 
 

Before the abovementioned functions could 

be applied, initial data transformations were 

necessary. The data was centered on mean and 

divided by a factor to produce more distinct values 

instead of the same 5 values for all questions. None 

of those operations changed the correlation of data 

prior to applying the functions.  

Based on the experience from model 1, the 

variables were sorted by correlation. To facilitate 

experimenting with the data model, a relative rank 

of correlation value was included in variable name 

while keeping the question number embedded. For 

example, variable V05_N19_20 had the fifth 

strongest correlation of all variables (05 in the 

name) and was a result of applying transformation 

(N in the name) to question Q19:20 answers 

(19_20 in the name). For some variables, an 

alternative version of the same variable was also 

used. For example V18b_A19_15 was an 

alternative variable to V18a_N19_15, containing 

answers to question Q19:15 without applying the 

function, with only the transformations described in 

data preparation section above. The decision on 

whether to include an alternative variable or not 

was made based on how much the correlation has 

improved as a result of applying a given function, 

with 0.05 as a threshold. However, models 

containing alternative versions of the variables 

were not any different from those using 

transformed variables and will not be presented 

here. The summary of functions applied to 

variables is presented below: 

 

 V01a_N19_3: q19_3 transformed using 
log(2, 1-X/4) 

 V02_N19_6: q19_6 transformed using 
power(X, 3) 

 V03_N19_6: q19_19 transformed using 
power(X, 3) 

 V04_A19_3: q19_10 transformed using X 

 V05_N19_20: q19_20 transformed using 
1/power(2,X) 

 V06a_N19_5: q19_5 transformed using 
exp(X) 

 V07_N19_18: q19_18 transformed using 
1/power(2,X) 

 V08_N19_22: q19_22 transformed using 
exp(X) 

 V09_N19_25: q19_25 transformed using 
sqrt(abs(X)) 

 V10N19_17: q19_17 transformed using 
1/power(X,3) 

 V11_N19_8: q19_8 transformed using 
sqrt(abs(X)) 

 V12a_N19_16: q19_16 transformed using 
1/power(2,X) 

 V13_N19_14: q19_14 transformed using 
1/X 

 V14_N19_2: q19_2 transformed using 
sqrt(abs(X)) 

 V15a_N19_12: q19_12 transformed using 
1/power(X,3) 

 V16_N19_7: q19_7 transformed using 
power(X, 2) 

 V17_N19_11: q19_11 transformed using 
1/power(X,3) 

 V18a_N19_15: q19_15 transformed using 
power(X, 3) 

 V19_N19_24: q19_24 transformed using 
1/X 

 V20_N19_23: q19_23 transformed using 
1/X 

 V21_N19_9: q19_9 transformed using 
sqrt(abs(X)) 

 

Figure 6: PLS regression model 2 - equivalent of model 1 with transformed variables 



120 
 

In case of untransformed variables, the regression 

model with the best fit used only 5 independent 

variables. Therefore, the analogical model was 

tried using transformed data. This time, the 

improvement in R
2
 was smaller and the model was 

able to explain 0.584 of the success. The resulting 

regression model is presented in Figure 6 above. 

 

Figure 7: PLS regression model with the best fit 

After some experimentation with various different 

models, R
2
=0.611 was achieved by one of them. It 

is presented in Figure 7. This was the best model 

found using either transformed or untransformed 

data. The variables used in this model are 

summarized as follows:  

 

 V14_N19_2: Overall, was it possible to 
define factors that were critical to how 

successful the project was? 

 V01a_N19_3: Did the end users use the data 
warehouse / BI solution provided? 

 V06a_N19_5: Was there a clear strategic 
vision for Business Intelligence or data 

warehouse? 

 V02_N19_6: Were the business needs that 
project tried to solve aligned to the 

abovementioned strategic vision? 

 V16_N19_7: Was project's underlying 
business case preceded by a business needs 

analysis? This does not mean your party 

(vendor or customer) performed this 

analysis as part of the project 

 V11_N19_8: Did the final solution evolve 
beyond initial scope? In case a program was 

established, answer on a program level 

 V04_A19_10: Did project scope definition 
concentrate on choosing the best 

opportunities ("low-hanging fruit")? 

 V17_N19_11: When an expert in a 
particular field was needed, was project 

team always granted access to an expert or 

enhanced by including one? 

 V15a_N19_12: When an expert was 
assigned to a team or made available to a 

team, was it always the best expert 

available? 

 V13_N19_14: If there were data quality 
issues at source systems, were they known 

before the start of BI initiative? 

 V12a_N19_16: Were there data quality 
issues in BI system or data warehouse? 

 V10N19_17: Were data quality issues in BI 
system or data warehouse eventually 

resolved? 

 V07_N19_18: Were technological issues 
encountered during the project? 

 V03_N19_19: If technological issues were 
present, were all of them resolved? 

 V05_N19_20: Were non-technological 
issues encountered during the project? 

 V08_N19_22: Data architecture was 
performed at the beginning of BI initiative 

(note - data architecture is different from 



121 
 

systems architecture or solution 

architecture) 

 V09_N19_25: Was there a process in place 
to measure business benefits? 

 

4.2 Measures Analysis  

 

From the three measures of data warehouse use 

proposed in the survey, none were found bad or 

even rather bad. Conversely, none of them were 

found definitely good. The best from the three was 

a weekly number of queries. The fact that it was 

positively evaluated in most responses and that 

mean value was close to being rather good makes 

this measurement method adequate and fairly 

universal. It is objective and can be automatically 

computed. 

Out of the two proposed measurement 

methods of access to subject matter experts, 

respondents found both “rather good”, with over 

two thirds of respondents agreeing with this view. 

Despite the fact that the mean response time turned 

out to score slightly better, we propose to use both 

as measurement for the “access to subject matter 

experts”- variable. To compute any of them, a 

formal process of request tracking should be made, 

but with this process in place, both measures are 

objective and easy to compute. 

Only one of the three proposed measurement 

methods for subject matter expert quality was 

found applicable. It turned out to be the same 

measurement method as for the access to subject 

matter experts. Because of that, the same 

arguments about measurement method objectivity 

and computability apply. 

From the three measures proposed in the 

survey for the question of whether there was 

enough business staff on the project, we found that 

both percentages of business issues resolved within 

a business week. The mean time to resolve a 

business issue qualify as an adequate measure for 

the variable was there enough business staff on a 

project. We found little difference in those 

measures as they are equally good and equally 

objective. They are both calculated in a similar way 

therefore either one of them could be used, or even 

both at the same time. 

Two measures of deliverable size in iterative 

development were proposed and we found that only 

a percentage of total project time for each iteration 

is a good one. However, this measure is only 

objective after the project ends. It can be calculated 

during the planning phases and project schedule 

can be used for that purpose. Unfortunately, this 

makes the measure less objective and easy to 

manipulate. This diminishes its applicability to 

predict initiative success. Despite these drawbacks, 

we postulate to use this measure in absence of 

better alternatives. Perhaps other measures could be 

examined in the future to replace this one, for 

instance percentage of total budget allocated or a 

similar measure. 

Out of three measures proposed for data 

quality issues at source systems, it was found that 

both counting the total number of issues and the 

number of unresolved issues are adequate and 

almost identically rated. Therefore no distinction 

can be made as to which one of them to use and we 

recommend using both. They should be easily 

computable if only the formal issue tracking 

process is in place. However, both measurement 

methods can only be used after the project end as 

the number of issues is unpredictable (following 

our earlier findings, a number of issues were 

discovered during the BI project). For this reason it 

would be hard to predict initiative success based 

only on the upfront knowledge about source 

systems data issues. 

From the three measurement methods 

proposed for data quality issues in BI system or 

data warehouse, no one better than the others were 

found. Therefore, it is conclude that this variable 

should be measured using the number of 

unresolved issues. The fact that this variable could 

be objectively measured only after the project ends 

makes it unsuitable for use in predicting project 

success. 

All three measurement methods proposed for 

benefits quantification were found adequate for 

quantification. Both ROI and IRR calculations 

were objective yet difficult to perform at project 

start, while relating capabilities to company 

strategy was less objective but easier to apply early 

in the project. It would be best if financial 

indicators could be calculated at project start; 

therefore those two methods would be preferred 

over the one called strategy, however all three are 

worth using and will probably work in practice. 

Both measurement methods proposed for 

business benefits definition were evaluated 

positively by survey respondents with percentage 

of business benefits objectively measureable being 

significantly better than just a written definition of 

the benefits. Both were objective and computable 

in the early stages of the project. Although, 

percentages were recommended as a better 

measurement, we also note that both are likely to 

be used in practice as it is unlikely that a company 

will have the benefits that are measureable without 

first defining them. Therefore, in practice, when the 

better measurement method can be applied, then so 

can the one that is worse off. 

From the two measurement methods proposed 

for business benefit measurements, only one was 

approved by survey respondents - the presence or 

absence of business benefits process owner. This 

measurement process can and should be used in 

predicting the project success. 

 



122 
 

 

Figure 8: Analysis of measurement methods of variables (Q22) 

 

For each of the variables analyzed in question 22, 

there was at least one measure that was deemed 

good enough to be generally used for all projects. 

However, almost no measure was found definitely 

good. There are two possible explanations for this 

phenomenon. Either there is no universally 

acceptable measurement method for those variables 

or such a variable exists but was not discovered in 

this study. Neither of the above possibilities can be 

verified using data gathered in this study. 

 

5. Hypotheses Analysis 

 

Hypothesis 1 (first part). It is possible to 

identify critical success factors (CSFs) of BI 

initiatives in large enterprises. 

 

To determine if factors exist that, if present, are 

more likely to lead to initiative success, an analysis 

of the correlation between the success and different 

factors surveyed was performed. To demonstrate 

causality, the statistical evidence of correlation 

between success and different key success criteria 

are complemented with an argumentation on what 

the casual relationship is. To show correlation 

between different criteria and the success of the 

initiatives, Spearman’s rank correlation 

coefficient (Spearman’s ρ) was chosen. 

Motivation for choosing Spearman’s rank 
correlation coefficient includes that it does not 

require the relationship to be linear, but also the 

fact that it is often used for Likert scales data sets 

in e.g. medicine, biochemistry and other sciences 

with success (Jamieson, 2004). The 

questions/factors with correlation coefficients with 

an absolute value of 0.25 or higher are (correlation 

coefficient in parenthesis): 

 

 Did project scope definition concentrate on 
choosing the best opportunities ("low-

hanging fruit")? (0,43) 

 Were non-technological issues encountered 
during the project? (-0,42) 

 Were the business needs that project tried to 
solve aligned to the abovementioned 

strategic vision? (0,41) 

 Was the project meant to address specific 
business needs of the sponsor? (0,38) 

 Data architecture was performed at the 
beginning of BI initiative (0,37) 

 Was there a clear strategic vision for 
Business Intelligence or data warehouse? 

(0,35) 

 Were there data quality issues at source 
systems? (-0,30) 

 If iterative development was chosen, did 
each iteration provide business value by 

itself or in connection with previous 

deliverables? (0,29) 

 Were technological issues encountered 
during the project? (-0,28) 

 Were data quality issues in BI system or 
data warehouse eventually resolved? (0,27) 

 Have business benefits been quantified? 
(0,26) 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%
Q

2
2

:1

Q
2

2
:2

Q
2

2
:3

Q
2

2
:4

Q
2

2
:5

Q
2

2
:6

Q
2

2
:7

Q
2

2
:8

Q
2

2
:9

Q
2

2
:1

0

Q
2

2
:1

1

Q
2

2
:1

2

Q
2

2
:1

3

Q
2

2
:1

4

Q
2

2
:1

5

Q
2

2
:1

6

Q
2

2
:1

7

Q
2

2
:1

8

Q
2

2
:1

9

Q
2

2
:2

0

Q
2

2
:2

1

Q
2

2
:2

2

Q
2

2
:2

3

Q
2

2
:2

4

Q
2

2
:2

5

Q
2

2
:2

6

Analysis of measurement methods of 

variables 

no answer 1 definitely yes 2 rather yes 3 neutral 4 rather not 5 definitely not



123 
 

 

From the above it can be conclude that there are in 

fact criteria (those factors with a large correlation 

with success of the initiative) that, when present, 

are associated with a higher likelihood of initiative 

success. 

 

Hypothesis 1 (second part). CSFs of BI 

initiatives are not the same as for other IS in 

large enterprises. 

 

Looking at the top criteria for successful initiatives 

with the highest ranking from the surveyed 

participants shows that success factors related to 

the project actually addressing a specific business 

need is most important. Building a solution that is 

tailored to user requirements and therefore enables 

usage of the Business Intelligence solution is also 

important. In the third place comes; documenting 

success criteria for the initiative early on in the 

initiative. 

Resolving technological issues, having a clear 

vision to guide the initiative and including a 

business need analysis early on are other important 

criteria. The survey clearly shows that non-

technological factors rank high amongst the 

answers.  To compare this to success factors for IS 

in general, a framework proposed by Delone and 

McLean (Delone & McLean, 1992; 2003) was 

used. The original framework proposed by these 

authors  has been updated with an updated version 

(Delone & McLean, 2003). As discussed 

previously the factors leading to the success of IS 

project in general include information quality, 

system quality and service quality. The first thing 

that strikes one in the framework proposed by 

Delone and McLean is the heavy focus on technical 

issues and factors, and little to none concentration 

on the softer factors that pertain to management 

issues. 

One difference from the general IS framework 

proposed by Delone and McLean and the findings 

from this study is the addition of more non-

technical factors such as the presence of a specific 

business need to be addressed by the initiative, a 

clear vision to guide the initiative and a pre-

initiative definition of success factors. 

 

Hypothesis 2. Non-technical and non-

technological CSFs play a dominant role in BI 

initiatives’ success in large enterprises. 
 

From what has been seen earlier, most of the CSFs 

with high correlation with success are in fact non-

technical. Examples are: 

 

1. Successful initiatives do focus, to a larger 
extent, on choosing the best opportunities 

i.e. “low-hanging fruit” or quick wins. 

2. Successful initiatives are more likely to 
have a clear strategic vision for Business 

Intelligence or data warehouse. 

3. Successful initiatives are more often meant 
to address a specific business need of the 

sponsor. 

4. The business needs addressed by successful 
initiatives are more often aligned with the 

abovementioned strategic vision. 

 

Looking at individual CSFs and the responses from 

the survey, one can see that for non-technical 

issues, there is a noteworthy difference between the 

results for successful initiatives and the non-

successful ones. 

One example of this is the area of iterative 

development. For those initiatives that used an 

iterative approach, each iteration was more likely 

to provide business value by itself for the 

successful ones. A total of 89% agreed to this, 

compared to 50% for the non-successful initiatives.  

Another example concerns the presence of a 

strategic vision for Business Intelligence. A total of 

81% of the successful initiatives had a clear 

strategic vision compared to only 46% of the non-

successful ones. 

In conjunction with the presence of a strategic 

vision for Business Intelligence as a whole, the 

successful initiatives are more likely to address a 

specific business need that is aligned to this 

strategic vision. Some 73% of the successful 

initiatives were addressing a specific business need 

aligned with the strategic vision. This is almost 

twice as common as the 39% of the non-successful 

ones. 

When looking at the scope definition, the 

successful initiatives are more often concentrating 

on choosing the best opportunities as depicted by 

the chart below. 57% of the successful initiatives 

do this compared to the non-successful ones 23%. 

All of the above examples are non-technical 

factors that are (to a varying degree) more 

commonly found in successful initiatives than 

those initiatives that fail. This in turn, indicates that 

non-technical and non-technological CSFs play a 

dominant role in BI initiatives’ success in large 
enterprises. 

 

Hypothesis 3. Technology plays a secondary 

role and is less critical for BI initiatives’ 
success in large enterprises 

 

The results from the survey show clearly that non-

technical factors were the hardest to solve. Results 

show that for all initiatives in the survey only 

slightly more than 20% found the technology 

related problems to be the hardest to solve. The fact 

that the non-technology related problems were 

harder to solve is even more evident when looking 

at the non-successful initiatives only, where almost 



124 
 

as many as 70% of the initiatives found the non-

technology related problem to be the hardest. 

Another view on this is the time spent on the 

different types of problems.  Again, as we can see a 

larger part of the time is spent on resolving non-

technological problems. This is especially true 

when looking at the non-successful initiatives, out 

of which almost 40% indicate that non-

technological issues took the most time. 

When comparing the successful initiatives 

with the non-successful ones and the presence of 

non-technological issues, it is clear that the 

initiatives that fail more often encounter non-

technological problems.  One interpretation of this 

could be that the presence of non-technological 

issues will have a detrimental effect on the 

likelihood of initiative success.  

 

Hypothesis 4. It is possible to develop 

objective measures for each of CSFs of BI 

initiatives in large enterprises. 

 

The result of this study shows that all the CSFs 

clearly have one or more measures that are ranked 

higher than the rest. The following summarizes the 

proposed measurements methods resulting from the 

survey. 

 

Area/Variable Proposed method(s) 

Data warehouse 

use 
 Weekly number of queries submitted 

Access to subject 

matter experts 
 Percentage of requests responded within one business day 

 Mean response time for a request 

Subject matter 

expert quality 
 Average number of questions needed to be asked per subject 

matter issue 

 Average number of available documents per subject matter issue 

 Percentage of questions answered within one business day 

Adequate level of 

business staff on 

the project 

 Percentage of business issues resolved within a business week  

 Mean time to resolve a business issue 

Size of 

deliverable in 

iterative 

development 

 Percentage of total project time for each iteration 

Data quality 

issues at source 

systems 

 Total number of issues  

 Number of unresolved issues 

Quality issues in 

BI system or data 

warehouse 

 Number of unresolved issues 

Benefits 

quantification 
 Calculated ROI 

 Proposition on how the business intelligence capabilities enables 
the realization of business strategy 

 Contribution to the internal rate of return 

Business benefits 

definition 
 Percentage of business benefits that are objectively measureable 

Business benefits 

measurements 
 Presence or absence of business benefits measurements process 

owner 

Table 1: Summary of variables analysis 

6. Conclusions 

 

The hypotheses of this study are focused on the 

factors that make Business Intelligence initiatives 

successful.  

A quantitative approach was used to gather 

data to support or reject the above hypotheses and a 

primary research survey was used for gathering 

data on the above hypotheses. The survey was 

conducted online during Q2 of 2011. The findings 

can be summarized as follows. 

 

6.1 Non-technological problems dominate in 

Business Intelligence projects 

 

Business Intelligence projects wrestle with both 

technological and non-technological problems, but 

the non-technological problems are found to be 



125 
 

both harder to solve sand more time consuming 

than their technological counterparts. For all 

initiatives in the survey, only slightly more than 

20% found the technology related problems to be 

the hardest. When looking at the non-successful 

initiatives only, almost as much as 70% found the 

non-technology related problem to be the hardest. 

The other view on this, the time spent on the 

different types of problems, also shows that a larger 

part of the time is spent on resolving non-

technological problems. This is especially true 

when looking at the non-successful initiatives, 

almost 40% of which indicate that non-

technological issues drove the most time. 

 

6.2 Successful projects share common factors 

 

Certain factors or attributes of Business 

Intelligence projects are correlated with the success 

of the initiative. The correlation has been 

calculated using Spearman’s rank and shown in 
the model that was developed using Partial Least 

Squares. By using rational argumentation, it can be 

shown that those factors do in fact have a causal 

relationship with the likelihood of success. The 

highest correlated factors are: 

 

 A project scope definition that concentrate 
on choosing the best opportunities ("low-

hanging fruit") 

 The alignment of the business needs that 
project tries to solve aligned with the 

strategic vision of business intelligence 

 A project that is meant to address specific 
business needs of the sponsor 

 Performing data architecture at the 
beginning of BI initiative 

 Having a clear strategic vision for Business 
Intelligence or data warehouse 

 

6.3 Successful and non-successful projects show 

differences in certain factors 

 

For a number of different factors, clear differences 

can be observed for successful and non-successful 

initiatives. This shows that successful initiatives are 

doing specific things more frequently than the non-

successful. Factors with great differences are: 

 

 The type of funding of the initiative 

 Business value provided by each iteration 

 An alignment of the business needs that 
project tried to solve aligned to the strategic 

vision of Business Intelligence 

 A project that is meant to address specific 
business needs of the sponsor 

 A project scope definition that concentrate 
on choosing the best opportunities ("low-

hanging fruit") 

6.4 Business Intelligence project have success 

factors that differ from IS projects in general 

 

Comparing the critical factors for BI projects that 

are correlated with success, or are otherwise more 

commonly found in successful initiatives, with the 

success factors for general IS projects found in 

literature shows that they do in fact differ. One of 

the major differences between the results found in 

this study is the focus on non-technological factors. 

One example of this are the frequently quoted 

papers by Delone and McLean (1992; 2003), that 

predominately discussed technological factors 

while the results of this study point to the non-

technological factors as being more important. 

 

6.5 Clear objective measures for CSFs can be 

identified 

 

Clear objective measures that are commonly agreed 

upon according to the survey exist for each of the 

CSFs analyzed in the survey. For some of the 

CSFs, the results are clearly pointing towards one 

of the suggested measures, while others can have 

multiple measures that claim to be equally good 

measures. However, due to practical reasons, to the 

limit of the survey size, measurements for only 10 

variables were proposed. For the remaining 

variables, it is impossible to verify whether the 

remaining variables do or do not have objective 

measurement methods.  

 

6.6 A statistical model can be constructed that, to a 

certain extent, predicts the success of BI projects 

 

A critical success factors framework for Business 

Intelligence initiatives have been developed as part 

of this study. The results from the survey show that 

61% of variability of success can be explained by 

changes in independent variables.  Limiting the 

model to 5 independent variables (out of 17 that 

offer the highest explanatory power of the model) 

makes the model only slightly less powerful, 

allowing it to explain 58% of success variability. A 

conclusion from this would be that managers need 

only to monitor properly and manage 5 factors to 

achieve better probability of success, than the 20%-

50% currently achievable in Business Intelligence. 

For a Business Intelligence project to be successful: 

 

1. Business Intelligence solution must be built 
with end users in mind, as they need to use 

it 

2. The Business Intelligence system needs to 

be closely tied to a company’s strategic 
vision 

3. Project needs to be properly scoped and 
prioritized to concentrate on best 

opportunities first 



126 
 

4. Although technological issues are 
encountered, all of them need to be solved 

5. Non-technological issues should be avoided 
as they can hinder the success of the BI 

initiative 

 

Below is a summary of the support for the four 

hypotheses of this study. 

 

Hypothesis 1. It is possible to identify critical 

success factors (CSFs) of BI initiatives and 

they are different from the CSFs of 

information systems in general. (supported). 

Hypothesis 2. Non-technical and non-

technological CSFs play a dominant role in BI 

initiatives’ success for initiatives (supported). 

Hypothesis 3. Technology plays a secondary 

role and is less critical for BI initiatives’ 

success for initiatives (supported). 

Hypothesis 4. It is possible to develop 

objective measures for each of CSFs of BI 

initiatives primarily (partially supported - the 

survey did not cover measurement analysis all 

CSFs). 

 

7. Future research 

 

The survey data received describes project 

primarily from Poland and primarily from a vendor 

perspective. Conducting a similar research on a 

more generalized sample spanning projects 

worldwide with equal emphasis on vendors and 

customers would be an obvious next step. It would 

verify if the relationship between success and its 

factors is a phenomenon local to Poland or rather, 

as we suspect, general in nature. 

Several other frameworks were reviewed in 

this study. A carefully constructed study with 

properly chosen variables would allow comparing 

each framework's validity with regards to 

contemporary Business Intelligence projects seeing 

if one is significantly stronger than the rest of them. 

This is not a trivial task, because of different 

treatment of success and its factors, using proxy 

variables for success or even modeling 

dependencies between various parts of a success. 

The word critical in its strict mathematical 

meaning would suggest that if even one of critical 

success factors is missing, success cannot happen. 

Such a relationship is impossible to model using 

linear regression. Even the application of data 

transformation must be taken into account, similar 

to the achieved linear combinations of some 

functions of variables. Therefore, another study 

could be performed to search for a model that 

would better explain the success using the variables 

used by us. It is possible that the variables used in 

this study would be able to predict more of the 

Business Intelligence success when organized in a 

different fashion 

Because almost 40% of success variability 

remains unexplained in this model, it is possible 

that it can be attributed to the weakness of the 

linear relationship assumed in the model, to the 

missing variable that remains unidentified or even 

to a random factor that is influenced only by 

chance and which cannot be controlled by 

management. Before a model with higher 

explanatory power is constructed, it is appropriate 

to keep looking for a missing variable that might 

potentially account for a significant portion of 

unexplained success variability. 

Although this work proposes several objective 

measurement criteria for independent variables, 

most of them cannot be considered universally 

applicable in all (or even most) Business 

Intelligence projects. Therefore, another study 

could be conducted to verify whether such 

universal and objective measures exist and attempt 

to define them. In case such measures do not exist, 

perhaps there is a set of measures for each variable 

such that at least one of the quantification methods 

would be universally applicable. 

One of the delimitations in this work was lack 

the treatment of NPV, ROI and other financial 

measures of Business Intelligence projects. Since 

success factors are different for BI than for IS in 

general, perhaps Business Intelligence managers 

could also employ different methods for calculating 

financial measures than in the case of other IS. In 

case such differences are identified, appropriate 

methods should be proposed and aforementioned 

differences should be uncovered.  

Refrences 

 

Adelman, S. 2003. Impossible Data Warehouse 

Situations. Solutions from the Experts. Boston, 

MA: Pearson Education. 

Adèr, & Mellenbergh, G. 2011. Advising on 

Research Methods: A consultant's companion. 

3
rd

 ed. Huizen, The Netherlands: Johannes van 

Kessel Publishing. 

Ariyachandra, T., & Watson, H. 2006. Which Data 

Warehouse Architecture Is Most Successful? 

Business Intelligence Journal , 11 (1). 

Babbie, E. 2009. The Practice of Social Research. 

Wadsworth Publishing. 

Beal, B. 2005. Report: Half of data warehouse 

projects to fail. Retrieved May 22, 2011, from 

Search CRM: 

http://searchcrm.techtarget.com/news/1066086/

Report-Half-of-data-warehouse-projects-to-fail 

Chen, L., Soliman, K., Mao, E., & Frolick, M. 

2000. Measuring user satisfaction with data 

warehouses: An exploratory study. Information 

& Management , 37 (3), 103-110. 

Daniel, D. 1961. Management Information Crisis. 

Harvard Business Review. 



127 
 

Delone, W., & McLean, E. 1992. Information 

Systems Success: The Quest for the Dependent 

Variable. Journal of Information System 

Research, 3 (1), 60-95. 

Delone, W., & McLean, E. 2003. The DeLone and 

McLean Model of Information Systems 

Success: A Ten-Year Update. Journal of 

Management Information Systems , 19 (4), 9-

30. 

Duncan, W. 1996. A Guide to the Project 

Management Body of Knowledge. Sylva, North 

Carolina: PMI Publishing. 

Ellison, R., & Moore, A. 2003. Trustworthy 

Refinement Through Intrusion-Aware Design. 

Carnegie Mellon Software Engineering 

Institute. 

Garthwaite, P. 1994. An Interpretation of Partial 

Least Squares. Journal of the American 

Statistical Association , 89 (425), 122-127. 

Hwang, M., & Xu, H. 2008. A Structural Model of 

Data Warehousing Success. Journal of 

Computer Information Systems , 48-56. 

Inmon, W. H. 2002. Building the Data Warehouse, 

3rd ed. New York, NY: John Wiley & Sons. 

Jamieson, S. 2004. Likert scales: how to (ab)use 

them. Medical Education, 1217-1218. 

Kimball, R., & Ross, M. 2002. The data warehouse 

Toolkit, 2 ed. New York, NY: John Wiley and 

Sons, Inc. 

Kimball, R., Ross, M., Thornthwaite, W., Mondy, 

J., & Becker, B. 2008. The Data Warehouse 

Lifecycle Toolkit, 2nd ed. Indianapolis, IN: 

Wiley Publishing. 

Laskowski, N. 2001. Gartner BI Summit: Business 

intelligence benefits lie in orchestration. 

Retrieved May 22, 2011, from 

http://searchbusinessanalytics.techtarget.com/ne

ws/2240035533/Gartner-BI-Summit-Business-

intelligence-benefits-lie-in-

orchestration?asrc=EM_NLN_13851353&track

=NL-544&ad=832107 

Legodi, I., & Barry, M.-L. 2010. The current 

challenges and status of risk management in 

enterprise data warehouse projects in South 

Africa. PICMET 2010 Technology Management 

for Global Economic Growth.  

Linstone, H. A., & Turoff, M. 2002. The Delphi 

Method Techniques and Applications.  

Markarian, J., Brobst, S., & Bedell, J. 2007. 

Critical Success Factors Deploying Pervasive 

BI. 

Meehan, P. 2011. Patrick Meehan speech during 

Gartner Business Intelligence Summit 2011. 

London, UK. Retrieved from 

http://link.brightcove.com/services/player/bcpid

1156010110?bctid=741282639001 

Merriam-Webster Dictionary. 2011. Retrieved May 

24, 2011, from Merriam-Webster.com 

Numminen, E. (2010). On the Economic Return of 

a Software Investment omManaging Cost, PhD 

Dissertation.  

Pirouz, D. 2006. An Overview of Partial Least 

Squares. Retrieved May 27, 2011, from Social 

Science Research Network: 

http://ssrn.com/abstract=1631359 

Pisello, T. 2001. IT Value Chain Management rom 

Social Science Research Network: htt Alinean, 

LLC. 

Popovic, A., & Jaklic, J. 2010. Benefits of Business 

Intelligence System Implementation: An 

Empirical Analysis of the Impact of Business 

Intelligence System Maturity on Information 

Quality. 

Ringle, C., Wende, S., & Will, S. 2005. SmartPLS 

2.0 (M3) Beta. Hamburg. 

Shin, B. 2003. An Exploratory Investigation of 

System Success Factors in Data Warehousing. 

Communications of the Association for 

Information Systems , 6 (4), 141-170. 

Solberg Søilen, K., & Hasslinger, A. 2009. How 

application integration, security issues and 

pricing strategies in business intelligence shape 

vendor differentiation. ECIS 2009: THIRD 

EUROPEAN COMPETITIVE INTELLIGENCE 

SYMPOSIUM, (pp. 252-260). Stockholm. 

Walton, E. J., & Dawson, S. 2001. Managers J., & 

Dawson, S. MPETITIVE INTELLIGENCE 

SYMPOSIUMtrategiesJournal of Management 

Studies, 173-179. 

Watson, H., Annino, D., Wixom, B., Avery, L., & 

Rutherford, M. 2001. Current Practices in Data 

Warehousing. Information Systems 

Management, 18 (1), 47-55. 

Wixom, B., & Watson, H. 2001. An Empirical 

Investigation of the Factors Affecting Data 

Warehousing Success. MIS Quarterly , 25 (1), 

17-41. 

Yeoh, W., & Koronios, A. 2010. Critical Success 

Factors for Business Intelligence Systems. 

Journal of Computer Information Systems, 50 

(3), 23-32. 

Yeoh, W., Gao, J., & Koronios, A. 2007. Towards 

a CSF Framework for Implementation of 

Business Intelligence Systems: A Delphi Study 

in Engineering Asset Management 

Organisations. Research and Practical Issues of 

Enterprise Information Systems II. Beijing, 

PRC.