18 A new evaluation model of ERP system success Abdesamad Zouine 1 Pierre Fenies 2 1 CRCGM Université d’Auvergne, France 2 CEROS, Paris X Nanterre, France Email: zouine.abdessamad@gmail.com Received March 10, accepted May 12 2015 ABSTRACT: This article presents a literature review about the success evaluation in the information system, and proposes a new evaluation success model suited to the ERP software. In the first part we present approaches, frameworks and models of the evaluation success previously used and empirically validated by researchers in the IS field. Then, we present our Evaluation Success Model, highlighting its three main theoretical foundations: Mathematical theory of communication, diffusion of innovation theory and adaptive structuration theory in the one hand, and we expose the main construct of this model named the ESF (Evaluation Success Factors) on the other hand. These factors are classified in three categories: technological, environmental and organizational evaluation factors. This work analyses articles published in the last decade about the success evaluation and delineates ten ESF’s widely used to evaluate the success of the ERP system project. KEYWORDS: ERP system, success, evaluation approaches, Evaluation Success Factors. Available for free online at https://ojs.hh.se/ Journal of Intelligence Studies in Business Vol 5, No 1 (2015) 18-39 mailto:zouine.abdessamad@gmail.com https://ojs.hh.se/ 19 1- Introduction The beginning of the 90’s was characterized by the emergence of ERP system, considered as one of the most important information systems software and the most expensive information technology project. The investment in this kind of project is under increasing scrutiny and pressure to justify their value and contribution to the performance, quality, and competitiveness of organizations (Gable et al., 2003). Currently, and after approximately two decades, all the largest business companies are now equipped with the ERP system in order to follow the environment change and business development. The integration of this project is considered as one of the most important challenges for the top management, project manager, ERP consultant and vendor at different levels of the organization. The ERP integration requires large investment, and it is associated with many problems in the implementation phase (Markus and Tanis, 2000). Despite the substantial investments made by organizations, its success had been minor (Davenport, 1998; Davis, 1989a; Gable et al., 2003; Sedera and Gable, 2010). In the literature review, many theoretical researches attempt to develop models to evaluate the information systems success. However, these models are not entirely appropriate for measuring ERP system success (Gable et al., 2003) for many reasons such as the specificities of the ERP system, its characteristics, and the complexity of implementation process. Organizations must support and manage the change introduced by the ERP system, because its integration needs an important reorganization and transformation in the business process, at both strategic and technical level. In many cases, this resistance is considered as a major risk of ERP project implementation. Before the evaluation of the ERP system, a framework has to be fixed and take into consideration the characteristics of the system. However, the context should dictate the appropriate specification and application of the ERP system (DeLone and McLean, 2003, 1992). Although, many success variables are proposed by researchers to evaluate the ERP success and attempt to explain the causal and the process model adapted to propose their constructs and measurement variables. This question about the causal and process model has been discussed in the literature about the IS evaluation. The process model suggests that an IS is first created, containing various features, which can be characterized as exhibiting various degrees of system and information quality. In contrast, a causal model studies the covariance of the success dimensions to determinate if there exist a causal relationship among them (DeLone and McLean, 2003). To evaluate their information systems, organizations require appropriate methods and tools; (Irani, 2002; Uwizeyemungu and Raymond, 2010) propose a new qualitative method for the ex-post evaluation of ERP system based on one hand on the organizational performance, and on the other hand on the automationed, informational and transformational effects that result from the integration and the use of the system. Their approach is based on a process model that takes into account at the same time practitioners’ dimension of evaluation, and researchers’ conception of evaluation that also can take two faces: qualitative or quantitative approaches of information system evaluation (Irani and Love, 2008). This phenomenon of IS evaluation is complicated and multifaceted; it must be examined from many perspectives (Song and Letch, 2012) and take into account different stakeholders involvement (Irani and Love, 2008; Irani et al., 2014; Stefanou, C.J., 2001). According to the ERP evaluation success, a new framework of ex-ante evaluation was proposed by (Stefanou, C.J., 2001) to evaluate the ERP system. This framework includes in the same time behavioral, technological and organizational perspective to evaluate the ERP software which is considered as a complex system (Irani, 2002; Stefanou, C.J., 2001). This step of success evaluation could be classified in the pre-implementation phase of the ERP integration process. It takes into account the selection process of the appropriate ERP software and all variables and criteria to select the most suitable one. The process of selection based on the one hand on both financial and non-financial approach and on the other hand it combines qualitative and quantitative measures (Stefanou, C.J., 2001). Relating to life-cycle product, the evolution of ERP integration process follows three phases: pre- implementation, implementation and post- implementation. However, in this study, we include both ex-ante and ex-post evaluation in to ERP success evaluation model because the evaluation is considered as a process that involves all ESF (Evaluation Success Factors) throughout the ERP life-cycle. This research paper will start with a presentation of the literature review about the different frameworks, models and approaches discussed by searchers in the IS Evaluation success (Davis, 1989b; DeLone and McLean, 2003, 1992; Gable et al., 2003; Ifinedo and Nahar, 2006; Irani and Love, 2008; Kaplan and Norton, 1992; Rosemann and Wiese, 1999; Seddon, 1997; Tsai et al., 2006) then, it will expose theoretical foundations based on three main theories: firstly, the mathematical theory of communication (Weaver and Shannon, 1949) used by Delone and McLean to develop their model about information success to explain the three levels that must be taken into account to evaluate IS success (technical level, semantic level and effectiveness level). Secondly, the diffusion of innovations theory (Rogers, 1983) mobilized by (Bradford and Florin, 2003) to explain the role of the diffusion of innovation on the ERP implementation success that will be used to involve and classify three principal factors in the conceptual 20 model: Technological; organizational and environmental. Thirdly, the Giddens’ theory of structuration (1984) to explain the interaction between the variables (Factors) and the performance in three levels: Individual workgroup and organizational performance. In the second part, we expose our conceptual model and highlight the principal evaluation success factors identified in both theoretical models and empirical studies. After that, we will explain how these ESF’s are classified taking into account the theoretical background in order to justify our conceptual perception. 2- Literature review about approaches, models and frameworks of ERP success This part focuses on the literature review on the research in IS success to summarize both theoretical backgrounds and empirical studies. The presentation will be chronologically respected in terms of frameworks, models and approaches developed in the IS field. Then, we will focus our attention on ERP as the main subject of this study. A review of different measurement approaches about the ERP success evaluation will be discussed to highlighting the importance of the measure in the information system and particularly the ERP software. 2-1- Frameworks of ERP evaluation success: Developing a framework is the first step in the evaluation success that must be appropriate to the features of the information system (Chand et al., 2005; Irani et al., 2014; Stefanou, C.J., 2001; Uwizeyemungu and Raymond, 2010) Many frameworks have been proposed taking into account several phases and dimensions of evaluation system success: strategic, tactical and operational levels. Generally, the framework explains eight categories: theoretical foundation, research approach, the object of analysis, unit of analysis, evaluation perspective, data gathering, data analysis and the methodology type (Urbach and Smolnik, 2008). 2-1-1- The CCP Framework: A CCP proposition could be considered as an important framework to assess the success of ERP system because this framework integrates three major dimensions of evaluation: Content, Context and Process (Irani and Love, 2008; Irani, 2002; Song and Letch, 2012). This new approach of evaluation answers three main questions: firstly, what is being measured (content) based on a socio-technological paradigm? Secondly, why and who of IS evaluation to be considered (Context)? And thirdly how will it be undertaken?. Many instruments could be used to answer this question like, cost benefits, ROI (Return on investment), User satisfaction that could be classified as an objective or subjective evaluation approach. This framework has been developed by (Irani and Love, 2008; Irani, 2002) to assist the managers and the decision makers in the process of the benefits evaluation of the IT/IS. They argue that there is not a good framework for assessing the impact of IS in the organization performance in the literature review and they added that there is no good framework for selecting the appropriate tools for IS investment. For these reasons, they try to propose a CCP framework to assess the cost and benefits of IS based on three constructs: Content, Context and Process. But we conclude after analyzing this framework that it is too large and general to be applicable to assess the success evaluation of the ERP system. 2-1-2- Stefanou’s framework: Another framework for the evaluation of the ERP system is developed by (Stefanou, C.J., 2001). It focuses on the pre-implementation phase. This framework named “ex-ante evaluation of ERP” assesses the selection process of ERP system and takes into account the complexity and the features of the ERP system. Both financial and non-financial approaches for ERP evaluation have been included in this framework. The financial approach is a traditional one used by the professionals to evaluate their IS success based on some financial indicators such as: return on investment, return on sales, cash flow, sales growth, inventory turnover, inventory level, operating income, asset utilization, capital budgeting, market share and shareholder value (Tsai et al., 2006). In contrast, in the case of ERP project, these financial indicators are not always reliable to assess the ERP impact because the benefits and the costs are not precisely identifiable, and they are not easily quantifiable (Stefanou, C.J., 2001). The second approach adopted by some researchers to evaluate the ERP success is based on a qualitative or subjective method that takes into consideration the intangible benefits such as: individual impact, learning and growth, consumer satisfaction, and the work group impact (Gable et al., 2003). 21 Stefanou’s framework consists of four phases: the first one considers the business vision as a point of starting for ERP integration. The second phase examines the business needs and the capabilities of the company to support and fit with the ERP system. The third phase requires the estimation of the costs and benefits for ERP system integration. The last phase refers to the analysis of issues involved in ERP operation, maintenance and evolution. Table 1: The potential costs and benefits associated with ERP life-cycle phases. (Stefanou, C.J., 2001) Phases of ERP life cycle Estimation of potential tangible and intangible costs, benefits and risk involved in each phase Phase 1: Business vision Risk associated with non-clarification of business vision and blurred business goals Phase 2a: Comparing needs versus capabilities and constraints Phase 2b: ERP selection Technological, organizational, human resources and financial capabilities and inefficiencies. Commitment to continuous change Costs/ benefits/risks associated with all-in-one or best-of-breed software options Costs/benefits associated with issues Costs involved in the selection process Phase 3: Implementation project Replacing of legacy systems Consulting fees Users training Implementation approaches Implementation partners Completion time Phase 4: Operation, maintenance and evolution Continuous re-engineering Software upgrades Additional functionality Benefits from ERP maturity both operational and strategic ERP users satisfaction Partner/customers satisfaction Business vision ERP selection ERP implementation ERP operation/ maintenance/ evolution Capabilities/ Constraints Evaluation of cost, benefits, risks: STRATEGIC-OPERATIONAL. Estimation of ROI/Value/Business case of ERP Phase 1 Requirements Phase 2 Phase 3 Phase 4 Figure 1: Major phases of ERP life-cycle (Stefanou, C.J., 2001) Analysis 22 Depending on the life-cycle approach developed above, a new proposition based on supply chain could be used to understand the different ERP project stakeholders. This approach of ERP success includes all the partners that operate in the integration process. This supply chain of the ERP success is based on three principal parts: firstly, the organization that considered as a client or customer; secondly the Software vendor (Vanilla ERP); thirdly the company of consulting or the integrator. The collaboration between all the different partners in the value chain is necessary. This alliance is one of the strategic benefits of the ERP implementation (Shang and Seddon, 2002). In this supply chain, the product is the ERP software, the consumer is the organization that will integrate it, and the external partners are the companies specialized in the ERP integration. But the question that arises here is: which are the parts that contribute in the success or the failure of the ERP system project? Analyzing this question from a SCM approach which considers the ERP as a product will be significant to determine the contribution of every partner in the ERP success. The quality of the product is one of the most important ESF in the project; in this case the quality of the product means the quality of ERP system. Many measures are proposed to assess this quality such as: Response time, convenience of access, realization of user requirements, correction of errors, security of data and models, integration of system, flexibility of the system, system efficiency, database contents, data currency, system accuracy and data accuracy (DeLone and McLean, 1992). Both vendor and consultant quality in terms of competencies is positively related to the ERP success (Ifinedo and Nahar, 2006). Some researchers consider this factor as an exogenous factor required in the ERP process success because all the partners came from the external environment. In the literature review about the ERP success evaluation, many studies include the vendor and consultant quality as an independent variable in their models to assess the ERP success (Bernroider et al., 2014; Ifinedo and Nahar, 2006; Tsai et al., 2012; Wang et al., 2008; Zhu et al., 2010). Some researchers found a significant and positive correlation between the VCQ (Vendor/Consultant Quality) and the success of the ERP system and they argue that it is important to take into account the competencies both strategic and technical of the partners in ERP system integration. The technical and knowledge transfers to the organization by the vendor and consultants are necessary to enhance the efficiency and the effectiveness of the ERP system in all phases of the project integration. For example, after the process of ERP system selection, the vendor transfers all the information about functionalities of the system, degree of customization, the functional coverage and other information supports to help the organization in the selection process. Combining both vendor and consultant in one factor is necessary because they are considered as an external source of expertise to the organization. (Ifinedo and Nahar, 2006; Sedera and Gable, 2010) found that vendor and consultant quality built a single factor “Knowledge Management Competencies”. The company that integrates an information system faces several starting conditions, according to competitive position, industry, financial position, size and structure (Markus and Tanis, 2000). However, these conditions may not be sufficient to explain clearly the success or the failure of the ERP system, but they have two principal impacts on enterprise system experience. Firstly, the strategic goals and plan may not be adapted to the ERP system specificities, this strategic alignment or fit will may be a problem for the organization in some cases. Secondly, the customization of the system could be necessary in many cases, that it means, starting conditions may not stay the same over the ERP experience (Markus and Tanis, 2000). ERP Supplier (ERP Editor) ERP Integrator (Consulting Company) Organization (Client or Consumer) ERP Success Figure 2: Process of the ERP Success 23 2-1-3- Soh and Markus framework: The ultimate goal of (Markus and Tanis, 2000; Soh and Markus, 1995) works is to create a new framework that enables a better understanding of the concept of ESS (Enterprise System Success). Answering these questions: how companies can succeed the integration of this technology? And what can be done to improve the chance of success? Authors define the success outcome as a multidimensional concept, a dynamic concept, and a relative one (to the concept of “optimal success”, representing the best an organization can hope to achieve with enterprise system). P. 184. The success can be defined by (Markus and Tanis, 2000) in terms of implementation project, or in terms of business results. The first definition answers the question: did the company succeed in getting the system up and running within some reasonable budget and schedule? The second answers the question: did the company succeed in realizing its business goals for the project? Based on the mergence process theories because (Markus and Tanis, 2000) consider that these theories combine both goals and actions with external forces and chance. They build their framework on a particular emergent process theory designed by (Soh and Markus, 1995) to explain how the enterprise system as a technology creates business value in organizations. 2-2- Models of ERP system success measurement Many models have been developed to evaluate the systems and technology’s success (Davis, 1989b; DeLone and McLean, 2003, 1992; Gable et al., 2003; Ifinedo and Nahar, 2006; Sedera and Gable, 2010; Shang and Seddon, 2002). These models have been validated empirically by many studies in information system. The results show that many case studies are investigated by applying the DeLone & McLean IS success model by using a structural equation modeling method (Dörr et al., 2013). However, these models assess the success in three levels of impact. The first one is an individual impact (Davis, 1989a) that sheds light on the users’ behaviors. The second level is the group impact (Gable et al., 2003; Sedera and Gable, 2010) interesting on the workgroup and its influence on the performance, and the third one is an organizational impact (DeLone and McLean, 1992). Although one model could assess more than one level of impact, for example, DeLone and McLean model take into account two levels of impact, individual and organizational performance. (Gable et al., 2003; Ifinedo and Nahar, 2006; Sedera and Gable, 2010) in their models about the ERP measurement success, they take into account three levels of impact, individual impact, workgroup impact and organizational impact to assess the success of ERP system. And finally Davis in his model of the technology Acceptance Model TAM, takes into account one level; the individual impact to assess the user perception and behavior. 2-2-1- Technology Acceptance Model TAM (Davis, 1989) This model has been widely used in the information system and considered as one of the main theoretical foundations (King and He, 2006). TAM has proven to be one of the most powerful models to explain user technology acceptance and users’ behavior (Wu et al., 2011). Davis claims that the technology usage is determined by two factors, Perceived usefulness and perceived ease of use, this individual impact is the main object of technology acceptance model. Many studies apply this model to understand the behavior and attitude of ERP system users and assess the satisfaction as a result of system use, the measurement of this satisfaction toward ERP system use is CSE Computer Self-Efficacy (Bradford and Florin, 2003; Kwahk and Lee, 2008; Scott and Walczak, 2009). Davis attempts to show that the user acceptance has been an impediment to the system information success; he considers that the User acceptance is the principal factor determining the success or failure of an information system project. For this reason, he investigates about why users accept or not an information technology and how users are influenced The IT use process Appropriate/Inappro priate use The competitive process Competitive position competitive dynamics IT impacts Organizational performance The IT conversion process IT expenditure IT management/ conversion activities IT assets Figure 3: Soh and Markus Framework (1995) 24 by the system features. To answer this question Davis develops his model based on Fishbein and Ajzen’s (1975) (Davis, 1989b) theory from psychology to explain the users’ attitudes and behaviors toward the information system use. To explain the system use, Davis’ investigation focuses on two main constructs, perceived usefulness and ease of use, which are theorized to be considered as determinants of system use (Davis, 1989a). The first construct is defined as “the degree to which a person believes that using a particular system would enhance his or her job performance”. The second construct is defined as “the degree to which a person believes that using a particular system would be free of effort”. The theoretical foundations for these two constructs were based on three main theories. Firstly, the Self-Efficacy from social cognitive theory (Bandura, 1999). Self-efficacy is considered as the foundation of human agency. The perceived self- efficacy occupies a pivotal role in social cognitive theory because its effects action are not only direct, but through its impact on other classes of determinants as well (Bandura, 1999). This concept of Self-efficacy has a causal relationship with motivation, performance and job satisfaction. Based on Bandura’s (1982) studies, (Davis, 1989b) explains both Self-efficacy judgment and the outcome judgments, and claims that the “outcome judgment” variable is similar to perceived of usefulness. The second theory used by Davis is the adoption of innovations theory from (Rogers, 1983). Davis outlines that the adoption of innovations suggests a prominent role for perceived ease of use. In the same vein, in their meta-analysis about the innovation characteristics and innovation adoption, implementation (Tornatzky and Klein, 1982) found that three innovation characteristics (compatibility, relative advantage, and complexity) had the most consistent significant relationships to innovation adoption. The third theory is the Cost-benefit paradigm from behavioral decision theory. It is relevant to perceived usefulness and ease of use (Davis, 1989b). Person choice among various decision-making strategies in terms of cognitive trade-off between the effort required employing the strategy and the quality of resulting decision, the distinction between subjective decision making performance and effort is similar to the distinction between the perceived usefulness and perceived ease of use (Davis, 1989b). System Perceived usefulness Perceived ease of use Attitude toward using Actual system use Figure 5: Technology acceptance model (Davis, 1989; p: 481) 25 Davis develops and validates new scales for two main variables, perceived usefulness and perceived ease of use, which are hypothesized to be the determinants of user acceptance technology. Based on two studies and 152 users as a sample of study, he developed items that were pretested for content validity and then tested for reliability and construct validity. In both studies, he finds that usefulness had a significantly greater correlation with usage behavior than did ease of use, and claims that the perceived ease of use is considered as an antecedent to perceived usefulness. However, after identifying two principal variables that impact the TAM construct: subjective norms and the mandatory use context. The updated of TAM named TAM 2 includes subjective norm as additional predictor of intention in the context of mandatory system use (Venkatesh and Davis, 2000). 2-2-2- DeLone and McLean Success Model: D&M model is the most cited model in information system success (Kronbichler et al., 2010; Sedera and Gable, 2010) it is one of the most famous models adopted by researchers to assess the success of information system in the last two decades. (Seddon, 1997) in his article named Respecification and Extension of D&M Model of IS, criticized this model about the inclusion of both causation and process interpretations, which lead to the confusion meanings that decrease the value of the model (Seddon, 1997). DeLone and McLean have up-dated their model based on these critics (DeLone and McLean, 2003). Despite, this update of their model, the first version stays the most adopted and most cited in the literature review in IS Success. The strength of D&M model resides in his theoretical foundation based on both Shannon & Weaver communication theory and Mason’s Communication systems approach (Mason, 1978; Weaver and Shannon, 1949). They claim that the information is considered as an output of an information system that can be measured at three principal levels: technical, semantic and effectiveness level, referring to the mathematical theory of communication (Weaver and Shannon, 1949) and its levels to analyze the message as a result of communication system. Defining and measuring the output of any system is always difficult, especially if the output is rather intangible. Information as an output is represented in symbolic form, this concept of signs is central to both information and communication; it is considered as the key link in the way one system affects another and thus involves the system’s context as well as the sign its self (Mason, 1978). Weaver classifies the problems of communication into three hierarchical levels A B and C: Level A. How accurately can the symbols of communication be transmitted? (The technical problem). Level B. How precisely do the transmitted symbols convey the desired meaning? (The semantic problem). Level C. How effectively does the received meaning affect conduct in the desired way? (The effectiveness problem). System Quality Information Quality User satisfaction Use Individual Impact Organizational Impact Figure 6: D&M IS Success Model (Delone & McLean, 1992) 26 D&M explain the concept of impact levels from communication theory and consider the serial nature of information as a form of communication. The information system is considered here as a sender that creates information which will be communicated to the recipient; this latter will be influenced by the content of this information. Following Mason’s scheme above (figure 7), the information system is considered as a production tool; the information is the product and the recipient is the user which is influenced by the content and quality of information. In the same vein D&M based on this approach they developed in their model two levels of influence or impact (individual and organizational). In this sense, they add that the flow of information throughout the production process to the use of information has an influence on individual and/or organizational performance. Based on these theoretical backgrounds, D&M developed six distinct categories or aspects of information system that become the constructs of their model, these constructs are: System Quality (SQ), Information Quality (IQ), USE, User Satisfaction, Individual Impact, and Organizational Impact. These variables are the most adopted to assess the success of an information system in the last two decades. However, the problem is the model construction that attempts to combine both causal and process explanations of IS Success (Seddon, 1997). The result of combining both variance and process model is that many boxes and arrows can have both a variance and an event in a process of interpretation, giving a sense of different parts of the model will cause slippage from one meaning for a box or arrow to another (Seddon, 1997), this later claims that the major difficulties with D&M model can be demonstrated by focusing attention on the Use as a construct. This box in (figure 6) can take three possible meanings: as a variable that proxies for the benefits from use, as the dependent variable in a variant model of future IS Use and thirdly as an event in a process leading to individual or organizational impact. In the Figure 8, Seddon shows the meaning of the categories in DeLone & McLean model of IS Success, and explains the combination of three models: Shanon and Weaver (1949) Technical Level Level Semantic Effectiveness or Influence Level Production Product Mason (1978) Receipt Influence on Recipient Influence on System Use System Quality Information Quality Categories of I/S Success User Satisfaction Individual Impact Organization al Impact Figure 7: Categories of IS Success (DeLone & McLean, 1992; p: 62) System Quality Information Quality Benefits from use implied by user satisfaction Benefits from use implied by IS use Benefits from use for Individuals Benefits from use for Organazations Figure 8: The meaning of the categories in D&M’s Model of IS Success; Seddon, 1997; p: 244 27  A variance model of IS Success, where the system quality and information quality are considered as an independent variables, and the dependent variables are the IS Use and User satisfaction.  The second model is a variance model of IS use as a behavior, that can take a second meaning for IS Use  The third model is a process model, where IS Use is considered as an event necessarily precedes the following constructs: User satisfaction, individual impact and organizational impact. (Seddon, 1997). Beyond the combination of both causation and process dimensions to explain the construction and the confusion in the meaning of the D&M model (Seddon, 1997) other considerations would take place such as the level taken into account to explain the success of an information system and the performance impacts. The IS Evaluation success is not limited to the internal factors as claimed by D&M in their Model based on Shannon and Weaver theory. For example the ERP system quality is not only a causal variable leading to success, but also can be considered as a result of other external factors such as organizational, innovation and environmental factors (Bradford and Florin, 2003; Ifinedo, 2011; Sedera and Gable, 2010). To answer for some critics considered troublesome, (DeLone and McLean, 2003) argue that their model is based on both process and causal considerations, the six dimensions of the model are interrelated rather than independent. Based on a process considered the first event of their model begins by creating an IS containing various specifies, the second event is the use of the system and its outputs. The final step is the impact result of this use on both individual and organizational performance. However, based on a causal dimension D&M explain the covariance between the independent and dependent variables to determine if there exists or not a causal relationship among the success dimensions. Combining taxonomy and success, this model was to help in the understanding of the possible causal interrelationships among the six dimensions of success. Despite the critics, D&M IS Success model stays one of the most adopted models in the information system field for two main reasons: its theoretical foundation and its empirical validation. But the question that arises is: Is the evaluation process of the IS Success based only on D&M model? Could it be possible to combine two theoretical models to assess the IS Success? What are the principal constructs of the combined model? What are the principal determinants of the ERP system Success? What are the theoretical foundations of this model? And what are the significant magnitudes of each factor in the model? 2-3- Evaluation approaches Many researchers tried to understand the relationship between the IT investments and the performance, emphasizing five main approaches to evaluate the IT projects (Bellaaj, 2010). These approaches are: Figure 9: Evaluation approaches of IS/IT - Evaluation approach based on the economic theory (Brynjolfsson, n.d.): the main goal of this approach is to understand the variance between the IT investment and the organizational productivity based on some economic criteria. - Evaluation approach based on social psychology (Davis, 1989a, 1989b; Venkatesh et al., 2003): beyond the economic approach, this one integrates the human factors as a determinant in the evaluation process of the IT investment and impact. Evaluation Approaches Evaluation approach based on the economic theory Evaluation approach based on social psychology Evaluation approach based on processes Evaluation approach based on the competitive analysis Evaluation approach based on the strategic alignment 28 - Evaluation approach based on the competitive analysis: this approach is developed by (Porter and Millar, 1985) explains how the technology affects all business. Authors’ outline that the information technology must be understood more than simple computers, it must be conceived of broadly to encompass the information that business create and use as well as a wide spectrum of increasingly convergent and linked technology that process the information, in their perception of the IT they adopt the concept of the value chain to explain the competitive advantages gained from the IT investments. - Evaluation approach based on the strategic alignment: This approach is developed by (Henderson and Venkatraman, 1993), it is widely used by the researchers in the information system to understand two main concepts; the first one is the fit between the information technology goals and the strategic objectives of the organization; the second is the functional integration (integration between business and functional domains). This approach suggests that the IT strategy must be coherent with the corporate strategy in order to improve the organizational performance. - Evaluation approach based on processes: a new conception of the IS success evaluation has been introduced by this approach based on emergent process theory developed by (Markus and Tanis, 2000; Soh and Markus, 1995). This approach highlights the inability of the economic model to evaluate the IS success, and proposes à new vision of evaluation based not only on the input evaluation (IT investment evaluation), but based also on the use and the impacts of the IT, under a creative process value. Three main approaches could be considered to evaluate the ERP System success; the first one is based on the financial criteria of performance (Nicolaou and Bhattacharya, 2006) to evaluate the ERP benefits (tangible benefits), the second approach is based on the non-financial approach to assess the intangible benefits of ERP System, and the last one is a mixed approach, for example to evaluate the ERP System, many perspectives of measurement must be taken into account such as the behavioral perspective (user acceptance), the strategic perspective (strategic alignment between Organizational goals and ERP), the economic perspective (cost, fees..) and the technological perspective (Organizational Fit and ERP system Integration). These four dimensions of ERP assessment were treated separately in the literature review about the ERP system success measurement. In this section, we will present two examples of evaluation approaches that synthesize the different evaluation perspectives mentioned above. Firstly we will propose an AHP approach to assess the ERP performance measures (Tsai et al., 2006). Secondly, we will present the balanced scorecard approach adopted largely by many searchers to evaluate the ERP System benefits (Chand et al., 2005; Rosemann and Wiese, 1999; Velcu, 2010). 2-3-1- AHP Approach of ERP performance assessment: The AHP approach (Analytic Hierarchy Process approach) consists in assessing the relative importance weights of ERP performance measurement; it can be used to select the main performance indicators of ERP system, and explains the contribution of ERP system in the organizational performance (Tsai et al., 2006). This approach is applied to decision-making problems to select the best and appropriate solution according to the importance of each alternative. In the case of ERP system two stages were presented by (Tsai et al., 2006) to assess the relative weights of ERP performance measurement. The first one consists of listing all the ERP performance measurement and evaluating their importance. The second stage focuses on constructing an AHP analysis framework and achieving the relative importance weights of 80 ERP performance measures by using a questionnaire with 7-point Likert-type scale (1=extremely unimportant, 7=extremely important). This approach focuses the post-implementation ERP stage. Based on D&M model 1992, this approach proposes a new taxonomy of performance measurement: the quality category, and the impact category of measurement. The quality concerns the ERP System, the information, System use and user satisfaction, the impact category concerns both individual and organizational levels. The result of this study shows that a company can select specific performance measurements according to three principal factors: goals of its ERP system, their needs and the specific context of the company. This means that every company must construct its key measurement performance taking into account the three main factors mentioned above. 2-3-2- Balanced Scorecard Approach of ERP performance measurement: This approach is developed by (Kaplan and Norton, 1992) to understand better and classify the performance measurements of the organization. They claim that the balanced scorecard allows managers to analyze the business performance from four main perspectives, financial perspective, internal business perspective, innovation/learning perspective and finally the customer perspective. This BSC framework is widely used in management science in different disciplines to assess the organizational performance. However, our attention focuses on the use of this approach to assess the performance introduced by the ERP system. Some researchers were interested in this question about assessing the ERP system performance from a BSC approach (Rosemann and Wiese, 1999; Velcu, 2010). They 29 explain how the BSC approach can be used to evaluate the business performance introduced by the ERP implementation on both operational and strategic levels. The aim objet of using the BSC approach is to explain the performance benefits that organizations gain from ERP system. This explanation follows four perspectives as defended by (Kaplan and Norton, 1992). After analyzing these perspectives, their application on the ERP system context appeared feasible and interesting to understand the performance beyond its traditional financial approach. The figure 9 explains how the ERP system contributes to the business performance from four different angles. This application of the BSC sheds some light on the understanding of three levels of ERP impact on the performance, the operational level, the tactical level and the strategic level. These levels provide a framework for analyzing benefits based on organizational strategy and ERP system goals throughout the ERP life-cycle. 3- Theoretical foundations: Firstly, we will present our conceptual model that is based on both theoretical and empirical background. This framework will be considered as a model of ERP system success evaluation that combine a causal and process considerations to assess the success of ERP project in three levels of performance: Individual performance, work group performance and organizational performance (Ifinedo and Nahar, 2006; Ifinedo, 2011; Ifinedo et al., 2010; Myers et al., 1997). The levels of analysis taken into account in this model were based on three theories: the first theory is the mathematical theory of communication as used by DeLone and McLean in there IS Success model to analyze the system quality and its impact on the information quality on the one hand and the impact of the information quality in users effectiveness on the other hand; the second theory is the innovation diffusion theory used to analyze and classify the different factors in three main boxes: Innovation factors, organizational factors and environmental factors; and finally the structuration theory to analyze the contribution of the ERP technology in the organizational performance. 3-1- Mathematical theory of communication The mathematical theory of communication (Mason, 1978; Weaver and Shannon, 1949) explains the interaction between three factors: the information system, the information as a product and the impact of the information on the individual and organizational performance. This approach is used by (DeLone and McLean, 1992) in their model of success to develop sex constructs considered as the •What are the best practices intruduced by the ERP system and how they contribute to improve the business value •How the ERP system improve the internal process by introducing a new busness process reengineering •What are the main criteria for assessing the user performance, user satisfaction, perceived ease of use and perceived usefulness of the ERP System •What are the appropriate indicators to measure the financial performance introduced by the ERP System Financial perspective Costumer perspective Innovation and learning perspective Internal perspective Figure 10: BSC perspectives for ERP performance evaluation 30 main variable to assess the success of the information system. 3-2- Innovation diffusion theory Based on the innovation diffusion theory, mainly the paradigm of variables determining the adoption of innovation (Rogers, 1983), three main factors appeared: Innovation/technological Factors, Environmental factors and Organization factors. In this taxonomy, each one of these factors can be explained in the ERP system context. These factors are extremely important in the ERP adoption phase, and they must be integrated in the process of the ERP system success (no success without technology adoption firstly). (Rogers, 1983) defines the constructs that constitute the perceived attributes of innovation in his paradigm of variables determining the adoption of technology as following: Compatibility: “Compatibility is the degree to which an innovation is perceived as consistent with the existing values, past experiences, and needs of potential adopters. An idea that is more compatible is less uncertain to the potential adopter. An innovation can be compatible or incompatible (1) with sociocultural values and beliefs, (2) with previously introduced ideas, or (3) with client needs for innovations”. Rogers, p: 223. Complexity “Is the degree to which an innovation is perceived as relatively difficult to understand and use” Rogers, p: 231 Relative advantage “Is the degree to which an innovation is perceived as being better than the idea it supersedes the relative advantage of an innovation, as perceived by members of a social system, is positively related to its rate of adoption”. Rogers, p 231 Trialability (System testing: during the final stages of ERP implementation, the project team should consider the inclusion of testing exercises as well as simulation before the system “goes live” (Al-Mashari et al., 2003; Finney and Corbett, 2007)) “Is the degree to which an innovation may be experimented with on a limited basis” Rogers, p: 231 Observability “Is the degree to which the results of an innovation are visible to others? The observability of an innovation, as perceived by members of a social system, is positively related to its rate of adoption”. Rogers, p: 231. All these constructs take place for determining the ERP system adoption as a new technological innovation introduced by the organization to improve its performance and achieve some strategic and operational goals. Taking into consideration these variables is an important step in the ERP system success process because we consider that there is no success outside the adoption of technology. When all the different stakeholders realize the usefulness and the perceived attributes of the ERP system, the ERP System adoption Perceived Attributes of ERP System Relative advantage Compatibility Complexity Trialability (System testing) Observability Type of Innovation Decision (ERP Implementation strategy decision) Communication Channels (Communication among stakeholders) Nature of the Social System (Legacy System) Extent of Change Agent’s Promotion Efforts (Management Change) Figure 11: Adopted from the paradigm of variables determining the adoption of innovation (M. Rogers, 1983 p: 233) 31 success and the quality of system begin to take place. Once adopted, the technology should bring productivity, efficiency, and satisfaction to individuals and organizations (DeSanctis and Poole, 1994). Main theoretical perspectives on technology and performance Characteristics of each perspective Examples of theoretical approaches Mathematical theory of communication Focus on both the information system and the information as an output in one hand, and explain their impact on the individual and organizational performance in the other. ( A process model) (Mason, 1978); (Weaver and Shannon, 1949) (See figure, 7) Innovation diffusion theory Focus on the technology adoption and use (A causal model) Paradigm of variables determining the adoption of innovation M. Rogers, 1983 (see Figure, 10) (Venkatesh et al., 2003) Structuration theory Focus on the interactions between actors and technology (a mixed model), and explain how the technology should bring productivity, efficiency, and satisfaction to individuals and organizations ACP (Adaptive structuration theory) approach (DeSanctis and Poole, 1994).(see Figure, 11) 3-3- Structuration theory (AST approach) Structuration theory associated with Giddens’ institutional theory of social evaluation has been largely applied to understand and explain organizational adoption of technologies (DeSanctis and Poole, 1994). We focus our attention only on the AST proposed by DeSanctis and Poole, 1994 to explain how the technology brings productivity, efficacy and satisfaction to both individuals and organizations. This approach which is based on the technology school was applied and explained by DeSanctis and Poole, 1994 in their Adaptive Structuration theory approach. The ASP is considered as a framework for studying variation in organization change and illustrating the impacts of advanced technologies on organizations. It was tested on a GDSS (Group decision support system) to answer the questions about how the technology affects people and organizations that use it, and how it improves workgroup performance. We consider this AST approach as an extension of the paradigm of variables determining the adoption of technology (see figure 11), because the adoption of technology is an important step in the appropriation process leading to improve the performance in three main levels (individual, group and organizational performance). (DeSanctis and Poole, 1994) outline the importance played by organizational members in the process to choose the most appropriate technology. Table 2: Theoretical perspectives 32 3-4- Evaluation Success Factors of the ERP system: 4- Journal Authors Geographic Area Sample size Evaluation phases of ERP system Sector Respondents Function Evaluation Success Factors ESF’s Journal of Research and Practice in Information Technology (Shih and Huang, 2009) Asia 165 ERP project Private End ERP users *Top management support *ERP system quality *System integration *ERP Fit Journal of computing in civil engineering (Chung et al., 2008) Asia 281 ERP project Private End ERP users *Top management support *Vendor and consultant quality *Information quality *ERP system quality International Conference on Information Systems (Gable et al., 2003) Australia 310 ERP project Public End ERP users *Individual implication *Information quality *ERP system quality Journal of information technology management (Ifinedo and Nahar, 2006) Asia 62 ERP project Private User/ Consultant/ Manager *Vendor and consultant quality *Work group implication *Individual implication *Information quality *ERP system quality Structure of technology: sophistication /Efficiency Organizational environment Decision outcomes: Efficiency/Quality /Performance Social interaction Knowledge and experience with the system Figure 12: Adapted from the AST constructs (Desanctis&Poole, 1994; p: 123) 33 Int. J. Production Economics (Ram et al., 2013b) (Ram et al., 2013a) (a) Australia 217 Implementat ion Private All levels of ERP users *Training and education *Business process reengineering *Project management *System integration *ERP fit Computers in Human Behavior (Ifinedo et al., 2010) Europe 109 Post- implementati on Private All levels of ERP users *Vendor and consultant quality *Work group implication *Individual implication *Information quality *ERP system quality Information & Management (Law and Ngai, 2007) Asia 96 ERP project Private All levels of ERP users *Business process reengineering Social and Behavioral Sciences (Candra, 2012) Asia 46 Implementat ion Private All levels of ERP users *Individual implication *ERP system quality *Information quality International journal of Information Management (Zhu et al., 2010) Asia 65 Post- implementati on Private CIO's/ Managers *Top management support *Vendor and consultant quality *Project management *System integration *ERP Fit Information & Management (Hong and Kim, 2002) Asia 105 Implementat ion Private ERP Project managers *Business process reengineering Information & Management (Velcu, 2010) Europe 88 Implementat ion Private CIO/CEO/CF O *Business process reengineering * Project management Computers in industry (Ehie and Madsen, 2005) USA 36 Implementat ion Private All levels of ERP users *Top management support *Vendor and consultant quality *Business process reengineering *Project management Information& Management (Bernroi der, 2008) USA 209 Post- implementati on Private All levels of ERP users *Business process reengineering *Information quality The Journal of strategic information system (Sedera and Gable, 2010) Asia 310 ERP project Private All levels of ERP users *Vendor and consultant quality *Individual implication *ERP system quality International journal of (Bradfor d and USA 51 Implementat ion Private ERP Managers *Top management support 34 Accounting Information Systems Florin, 2003) *Training and education *Business process reengineering *System integration *ERP Fit Journal of Manufacturin g Systems (Chou and Hong, 2013) Asia 117 Implementat ion Private ERP users *Vendor and consultant quality *Individual implication Information quality *Information quality *ERP system quality The Journal of Systems and Software (Ifinedo, 2011) Europe 109 ERP project Private ERP users *Individual implication Information quality *Information quality *ERP system quality International journal of project management (Ram et al., 2013a) (Ram et al., 2013b) (b) Australia 209 Post- implementati on Private Senior ERP managers *Training and education *Business process reengineering *Project management *System integration *ERP fit The Journal of Systems and Software (Wang et al., 2008) Asia 90 Implementat ion Private CIO’s *Vendor and consultant quality *Top management support *Project management *Information quality Computers in Human Behavior (Yoon, 2009) Asia 152 Private ERP senior managers *Information quality International journal of human computer studies (Choi et al., 2007) Asia 223 Learning Educatio n Students *Training and education *ERP system quality Information& Management (Scott and Walczak, 2009) USA 234 Learning Educatio n Students *Top management support *Vendor and consultant quality Computers in Human Behavior (Amoako - Gyampa h, 2007) USA 278 Implementat ion Private End users *ERP system quality Decision support systems (Wang and Chen, 2006) Asia 122 ERP project Private ERP/IT Managers *System integration *ERP Fit International Journal of production economics (Chien et al., 2007) Asia 139 ERP project Private Senior ERP managers *Project management 35 Information& Management (Tsai et al., 2012) Asia 278 Implementat ion Private ERP users *Vendor and consultant quality *Individual implication *ERP system quality Information& Management (Kwahk and Lee, 2008) Asia 273 Post- implementati on Private ERP users *Top management support *ERP system quality Decision support systems (Chou and Chang, 2008) Asia 166 ERP project Private ERP users *Organizational factors Computers in Human Behavior (Grant et al., 2013) USA 122 Implementat ion Private ERP users *Organizational factors *System integration *ERP Fit Information& Management (Sun et al., 2009) Asia 138 ERP project Private ERP users *ERP system quality International journal of project management (Bernroi der et al., 2014) Europe 209 Implementat ion Private ERP users *Top management support *Vendor and consultant quality *Training and education *Business process reengineering *system integration *ERP Fit The Service Industries Journal (Lapiedr a et al., 2011) Europe 134 Implementat ion Private ERP users *Vendor and consultant quality 4-1- Evaluation ERP system success model: Organizational Factors: *Top Management Support *Project Management *Individual Implication * Work Group implication Technological/Innova tion Factors: *BPR *ERP Fit *System integration *System configuration and Customization Environmental Factors: *Vendor Competencies *Consultant Competencies *Training and education *Knowledge Transfers Information/Data Quality ERP System Quality Individual Performan ce Group Performan ce Organizational Performance Benefits E R P S u c c e ss 36 5- Discussion and implications Research implications: This study provides both theoretical backgrounds and empirical contribution to understand the factors that impact the ERP project success, this impact was measured in three levels of performance, individual, group and organization. Thus, this study proposes a new taxonomy of the evaluation success factors and explains the ERP system success process using a strong theoretical foundations, mathematical theory of communication, diffusion innovation theory and AST (Adaptive Structuration theory). The theoretical model developed in this work explains the ERP system success from two main dimensions, a causal dimension and a process dimension. The first one highlights the variables that contribute on the ERP system adoption and use, based on diffusions of innovations theory (Rogers, 1984). The second sheds the light on the process of the ERP system success through the explanation of interaction between organizational, individual and technological variables based on the one hand on the mathematical theory of communication to explain how the system quality output impacts the individual and organizational performance (Mason, 1978; Weaver and Shannon, 1949), and on the other hand on the AST (DeSanctis and Poole, 1994) to explain the interaction between the human actors (ERP Users) and the technology, and how this later leads to improve the efficiency, quality and performance. However, we exposed the main frameworks, approaches and models interested on the ERP system success and measurement in the literature. Thereby, we explained the feasibility and the fit of each one of these theoretical backgrounds to be applied to evaluate the success of the ERP system project including the specificities and implementation phases of the software. The theoretical model developed in this study is appropriated to the ERP system; it takes into account the features of both implementation and use of the ERP system. Because, the ERP system is considered as a project including different stakeholders, Organization involvement, user involvement, vendor and consultant involvement, it success depends on the collaboration between all the organization partners. Thus, the model explains how the Organizational, technological and environmental critical factors contribute to the ERP system adoption and use, which considered as a synonymous of the ERP system quality. Then, the model shows the quality output represented by data and information quality, and how this later affects the performance and the efficiency. The definition of the success adopted in this model reveals that the success is considered as a correspondence and an interaction (Lyytinen and Hirschheim, 1987), 1987). The correspondence highlights the fit between the ERP system and the organization objectives that leads to improve the organizational performance. The interaction success represents the positive user attitudes toward the ERP system, which contribute to improve both individual and workgroup performance. Managerial implications: This research work provides a new tool to practitioners enabling them a better understanding of the ERP system success project. Information system managers, top management and ERP users need to understand the implication of their actions in the success process and how they contribute in the performance improvement. Thus, this work seeks to highlight the vendor and consultants contributions to perform the ERP project. To face more than three quarters of unsuccessful ERP project, organizations need to be able to evaluate their information system projects. This need leads us to investigate this question by developing a new model that explain the relationships between the ERP partners on the one hand and propose the main evaluation factors to assess the ERP project success. 6- Conclusion This attempt to develop a new model of ERP system success evaluation is motivated by the need of companies to justify and understand their investments in this kind of information technology project. ERP system project should not be considered only as a top management project but an organizational project that integrates all the actors and stakeholders, for this reason in our model of ERP system success evaluation we take into consideration the role of all partners and actors for different level of analysis and different phases of ERP project integration. Three categories of evaluation factors were proposed: organizational factors, environmental factors and technology factors. These factors are crucial to evaluate the success of the ERP system project; they contribute considerably to understand the process of the ERP system success. Organizations should give more attention to these factors to succeed their information system project and to get a high quality system, accepted and used by employees. As highlighted in our model the success should be evaluated from three main levels of analysis: individual level, group level and organizational level. This model combines two principal conceptions of the success concept, the first one coming from the Delone & MaClean model to understand the main variables of the ERP system success and give more importance to the technological aspect based on the quality system as the principal starting point of the success process. However, TAM model give more importance to the human factor in the technology success based on the acceptance and use as two main criteria of the system success. But, nether on nor the other outline the external factors that contribute to the success of this project, it seems that these exogenous factors are important in the ERP system project acceptance as a new technology introduced by organizations. Theoretical basis of these factors derived from the diffusion of innovation theory (Rogers, 1983) that 37 outlines the importance of the environmental, technological and organizational factors in the technology adoption. This work proposes a set of tools to evaluate the ERP project success, many approaches models and framework were proposed to understand the evaluation success process. Summarizing works previously presented in the literature review about the success evaluation of the ERP system project. The next step for our work is to validate empirically our model. References Al-Mashari, M., Al-Mudimigh, A., Zairi, M., 2003. Enterprise resource planning: A taxonomy of critical factors. European Journal of Operational Research 146, 352–364. Amoako-Gyampah, K., 2007. Perceived usefulness, user involvement and behavioral intention: an empirical study of ERP implementation. Computers in Human Behavior 23, 1232–1248. Bandura, A., 1999. Social cognitive theory: An agentic perspective. Asian Journal of Social Psychology 2, 21–41. Bellaaj, M., 2010. Technologies de l’information et performance organisationnelle : différentes approches d’évaluation. (Manuscrit). Bernroider, E.W.N., 2008. IT governance for enterprise resource planning supported by the DeLone–McLean model of information systems success. Information & Management 45, 257– 269. Bernroider, E.W.N., Wong, C.W.Y., Lai, K., 2014. From dynamic capabilities to ERP enabled business improvements: The mediating effect of the implementation project. International Journal of Project Management 32, 350–362. Bradford, M., Florin, J., 2003. Examining the role of innovation diffusion factors on the implementation success of enterprise resource planning systems. International Journal of Accounting Information Systems 4, 205–225. Brynjolfsson E. et Hitt L. (1996), “Paradox lost? Firm-level evidence on the returns to information systems spending”, Management Science, 42, 541-558 Candra, S., 2012. ERP Implementation Success and Knowledge Capability. Procedia - Social and Behavioral Sciences 65, 141–149. Chand, D., Hachey, G., Hunton, J., Owhoso, V., Vasudevan, S., 2005. A balanced scorecard based framework for assessing the strategic impacts of ERP systems. Computers in Industry 56, 558– 572. Chien, S.-W., Hu, C., Reimers, K., Lin, J.-S., 2007. The influence of centrifugal and centripetal forces on ERP project success in small and medium-sized enterprises in China and Taiwan. International Journal of Production Economics 107, 380–396. Choi, D.H., Kim, J., Kim, S.H., 2007. ERP training with a web-based electronic learning system: The flow theory perspective. International Journal of Human-Computer Studies 65, 223–243. Chou, J.-S., Hong, J.-H., 2013. Assessing the impact of quality determinants and user characteristics on successful enterprise resource planning project implementation. Journal of Manufacturing Systems 32, 792–800. Chou, S.-W., Chang, Y.-C., 2008. The implementation factors that influence the ERP (enterprise resource planning) benefits. Decision Support Systems 46, 149–157. Chung, B.Y., J. Skibniewski, M., C. Lucas Jr, H., Kwak, Y.H., 2008. Analyzing Enterprise Resource Planning System Implementation Success Factors in the Engineering–Construction Industry. Journal of Computing in Civil Engineering 22, 373–382. Davenport, T.., 1998. Putting the Enterprise into the Enterprise System. Harvard Business Review 76, 121–131. Davis, F.D., 1989a. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13, 318–340. Davis, F.D., 1989b. User acceptance of computer technology: a comparison of two theoretical models. Management science 35, 982–1003. DeLone, W.H., McLean, E.R., 1992. Information Systems Success: The Quest for the Dependent Variable. Information Systems Research 3, 6.–95. DeLone, W.H., McLean, E.R., 2003. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. The D e Lone an D M c Lean Mo D e L of Infor M a TI on Sy ST e M Succe SS 9 Journal of Management Information Systems 19, 9–30. DeSanctis, G., Poole, M.S., 1994. Capturing the Complexity in Advanced Technology Use: Adaptive Structuration Theory. Organization Science 15, 121–147. Dörr, S., Walther, S., Eymann, T., 2013. Information Systems Success - A Quantitative Literature Review and Comparison. Presented at the 11th International Conference on Wirtschaftsinformatik, Germany. Ehie, I.C., Madsen, M., 2005. Identifying critical issues in enterprise resource planning (ERP) implementation. Computers in Industry 56, 545– 557. Finney, S., Corbett, M., 2007. ERP implementation: a compilation and analysis of critical success factors. Business Process Management Journal 13, 329–347. Gable, G.G., Sedera, D., Chan, T., 2003. Enterprise systems success: a measurement model, in: Proceedings of the 24th International Conference on Information Systems Association for Information Systems. Presented at the Proceedings of the 24th International Conference on Information Systems Association for Information Systems, Seattle, Washington, pp. 576–591. Grant, D., Hwang, Y., Tu, Q., 2013. An empirical investigation of six levels of enterprise resource planning integration. Computers in Human Behavior 29, 2123–2133. 38 Henderson, J.C., Venkatraman, N., 1993. Strategic alignement: Leveraging information technology for transforming organizations. IBM System Journal 38, 472–484. Hong, K.-K., Kim, Y.-G., 2002. The critical success factors for ERP implementation: an organizational fit perspective. Information & Management 40, 25–40. Ifinedo, P., 2011. Examining the influences of external expertise and in-house computer/IT knowledge on ERP system success. Journal of Systems and Software 84, 2065–2078. Ifinedo, P., Nahar, N., 2006. Quality, Impact and Success of ERP Systems: A Study Involving Some Firms in the Nordic-Baltic Region. Journal of Information Technology Impact 6, 19–46. Ifinedo, P., Rapp, B., Ifinedo, A., Sundberg, K., 2010. Relationships among ERP post- implementation success constructs: An analysis at the organizational level. Computers in Human Behavior 26, 1136–1148. Irani, Z., 2002. Information systems evaluation: navigating through the problem domain. Information & Management 40, 11–24. Irani, Z., Love, P.E.D., 2008. Evaluating information systems Public and private sector. Elsevier. Irani, Z., Sharif, A., Kamal, M.M., Love, P.E.D., 2014. Visualising a knowledge mapping of information systems investment evaluation. Expert Systems with Applications 41, 105–125. Kaplan, R.S., Norton, D.P., 1992. The Balanced Scorecard – Measures that Drive Business Perfomance. Harvard Business Review January- February, 71–79. King, W.R., He, J., 2006. A meta-analysis of the technology acceptance model. Information & Management 43, 740–755. Kronbichler, S.A., Ostermann, H., Staudinger, R., 2010. A comparison of ERP-Success measurment approaches. Journal of Information Systems and technologie Management 7, 281–310. Kwahk, K.-Y., Lee, J.-N., 2008. The role of readiness for change in ERP implementation: Theoretical bases and empirical validation. Information & Management 45, 474–481. Lapiedra, R., Alegre, J., Chiva, R., 2011. The importance of management innovation and consultant services on ERP implementation success. The Service Industries Journal 31, 1907– 1917. Law, C.C.H., Ngai, E.W.T., 2007. ERP systems adoption: An exploratory study of the organizational factors and impacts of ERP success. Information & Management 44, 418– 432. Lyytinen, K., Hirschheim, R., 1987. Information systems failures - a survey and classification of the empirical literature. Oxford Surveys in Information Technology 4, 257–309. Markus, M.L., Tanis, C., 2000. the enterprise system experience from adoption to success. Mason, R.O., 1978. Measuring information output: A communication systems approach. Information & Management 1, 219–234. Myers, B.L., Kappelman, L.A., Prybutok, V.R., 1997. A Comprehensive Model fo Assessing the Quality and Productivity of the Information System Function: Toward A Theory fo Information Systems Assessment. Information Recources Management Journal 10, 6–25. Nicolaou, A.I., Bhattacharya, S., 2006. Organizational performance effects of ERP systems usage: The impact of post- implementation changes. International Journal of Accounting Information Systems 7, 18–35. Porter, M., Millar, V., 1985. How Information Gives You Competitive Advantage. Harvard Business Review 149–160. Ram, J., Corkindale, D., Wu, M.-L., 2013a. Implementation critical success factors (CSFs) for ERP: Do they contribute to implementation success and post-implementation performance? International Journal of Production Economics 144, 157–174. Ram, J., Wu, M.-L., Tagg, R., 2013b. Competitive advantage from ERP projects: Examining the role of key implementation drivers. International Journal of Project Management. Rogers, E.M., 1983. DIFFUSION OF INNOVATIONS, 3rd ed. Free Press, United States of America. Rosemann, M., Wiese, J., 1999. Measuring the Performance of ERP Software – a Balanced Scorecard Approach, in: Proc. 10th Australasian Conference on Information Systems. Presented at the Proc. 10th Australasian Conference on Information Systems, pp. 773–784. Scott, J.E., Walczak, S., 2009. Cognitive engagement with a multimedia ERP training tool: Assessing computer self-efficacy and technology acceptance. Information & Management 46, 221– 232. Seddon, P., 1997. A Respecification and Extension of the DeLone and McLean Model of IS Success. Information Systems Research 8, 240–253. Sedera, D., Gable, G.G., 2010. Knowledge Management Competence for Enterprise System Success. The Journal of Strategic Information Systems 19, 296–306. Shang, S., Seddon, B.P., 2002. Assessing and managing the benefits of enterprise systems: the business manager’s perspective. Information Systems Journal 12, 271–299. Shih, Y.-Y., Huang, S.-S., 2009. The Actual Usage of ERP Systems: An Extended Technology Acceptance Perspective. Journal of Research and Practice in Information Technology 41, 263–276. Soh, C., Markus, M.L., 1995. How it creates business value: A process theory synthesis, in: Proc. International Conference on Information Systems. Presented at the Proc. International Conference on Information Systems. Song, X., Letch, N., 2012. Research on IT/IS Evaluation: A25 Year Review. The Electronic 39 Journal Information Systems Evaluation 15, 276– 287. Stefanou, C.J., C., 2001. A framework for the ex- ante evaluation of ERP software. European Journal of Information Systems 204–2015. Sun, Y., Bhattacherjee, A., Ma, Q., 2009. Extending technology usage to work settings: The role of perceived work compatibility in ERP implementation. Information & Management 46, 351–356. Tornatzky, L.G., Klein, K.J., 1982. Innovation Characteristics and Innovation Adoption- implementation: A Meta-analysis of findings. IEEE Transactions on engineering management 29, 28–43. Tsai, W.-H., Hsu, P.-Y., Cheng, J.M.-S., 2006. An AHP approach to assessing the relative importance weights of ERP performance measures. International Journal of management & enterprise development 3, 351–375.Tsai, W.-H., Lee, P.-L., Shen, Y.-S., Lin, H.-L., 2012. A comprehensive study of the relationship between enterprise resource planning selection criteria and enterprise resource planning system success. Information & Management 49, 36–46. Urbach, N., Smolnik, S., 2008. A Methodological Examination of Empirical Research on Information Systems Success: 2003 to 2007. Presented at the Proceedings of the Fourteenth Americas Conference on Information Systems, Toronto Canada. Uwizeyemungu, S., Raymond, L., 2010. Linking the Effects of ERP to Organizational Performance: Development and Initial Validation of an Evaluation Method. Information Systems Management 27, 25–41. Velcu, O., 2010. Strategic alignment of ERP implementation stages: An empirical investigation. Information & Management 47, 158–166. Venkatesh, V., Davis, F.D., 2000. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science 46, 186–204. Venkatesh, V., Morris, M.G., Davis, G.B., Davis, F.D., 2003. USER ACCEPTANCE OF INFORMATION TECHNOLOGY: TOWARD A UNIFIED VIEW1. MIS Quarterly 27, 425–478. Wang, E.T.G., Chen, J.H.F., 2006. The influence of governance equilibrium on ERP project success. Decision Support Systems 41, 708–727. Wang, E.T.G., Shih, S.-P., Jiang, J.J., Klein, G., 2008. The consistency among facilitating factors and ERP implementation success: A holistic view of fit. Journal of Systems and Software 81, 1609– 1621. Weaver, W., Shannon, C., 1949. The mathematical theorie of communication. Wu, K., Zhao, Y., Zhu, Q., Tan, X., Zheng, H., 2011. A meta-analysis of the impact of trust on technology acceptance model: Investigation of moderating influence of subject and context type. International Journal of Information Management 31, 572–581. Yoon, C., 2009. The effects of organizational citizenship behaviors on ERP system success. Computers in Human Behavior 25, 421–428. Zhu, Y., Li, Y., Wang, W., Chen, J., 2010. What leads to post-implementation success of ERP? An empirical study of the Chinese retail industry. International Journal of Information Management 30, 265–276.