Microsoft Word - 18-3762_s1_ETASR_V10_N5_pp6242-6248 Engineering, Technology & Applied Science Research Vol. 10, No. 5, 2020, 6242-6248 6242 www.etasr.com Khand & Kalhoro: Testing and Validating DeLone and MacLean IS Model: ERP System Success in … Testing and Validating DeLone and MacLean IS Model: ERP System Success in Higher Education Institutions of Pakistan Zahid Hussain Khand Department of Computer Science Sukkur IBA University Sukkur, Pakistan zahid@iba-suk.edu.pk Muhammad Ramzan Kalhoro Department of Business Administration Sukkur IBA University Sukkur, Pakistan ramzan.kalhoro@iba-suk.edu.pk Abstract−The universities in Pakistan have recently started replacing their old legacy systems with ERP systems which are commonly used in business organizations to gain a competitive edge over competitors. The Higher Education Commission (HEC) has implemented ERPs systems at eight different universities of Pakistan. HEC has invested a huge amount of money to facilitate the integration, customization, and implementation of ERP systems in these universities. Previous studies have mainly focused on Critical Success Factors (CSFs) and risk factors of ERP systems. This study intends to empirically measure the support of ERP systems in teaching processes. DeLone and McLean model of Information Systems (IS) success is one of the most commonly used models cited in IS literature. In this study, DeLone and McLean model was applied at a University level analysis to access the impact of ERP in higher educational institutions and their support in the improvement of academic processes. Hypotheses were tested on the research model using empirical data collected from 230 respondents, including students and faculty at two selected universities, with the use of a questionnaire. The data were analyzed by structural equation modeling. The model was empirically tested and the findings showed that use and user satisfaction effected most net benefits. Information quality, system quality and service quality accounted for 42.6% effect in use. Whereas, information quality, system quality, service quality and use accounted for 46.5% variance in user satisfaction. Keywords-higher education; Enterprise Resource Planning (ERP) system; DeLone and McLean IS model I. INTRODUCTION The term ERP (Enterprise Resource Planning) was coined in early 1990s. An ERP is a multi-module application software package, which tightly integrates data and business processes for sharing information in an organization [1]. Higher educational institutions have implemented ERP systems to automate and customize their business processes, which include student records, admissions, student finances, and most common academic and administrative services [2]. ERP systems are implemented in higher education to improve the performance and the efficiency of the organization to cope with the changing environment. Despite that ERP promises to benefit the organizations and even though it is a substantial capital investment, it is not necessarily the case that all ERP systems implementations have successful outcomes. In response to that demand, higher education institutions in Pakistan have started implementing ERP systems to gain a competitive edge over the competitors and enhance the efficiency in the processes. ERP has automated and integrated the key processes related to academic and administrative services [3]. The Higher Education Commission (HEC) of Pakistan has invested a significant amount of funding to facilitate the customization and implementation of ERPs in eight selected universities (Dow University of Health Sciences, Karachi (DUHS), University of Engineering and Technology, Peshawar (UET), Quaid-e-Azam University, Islamabad (QAU), Islamia University, Bahawalpur (IUB), Balochistan University of Information Technology and Management Sciences, Quetta (BUITEMS), University of Punjab, Lahore (PU), Sukkur Institute of Business Administration (SIBA), and Institute of Business Administration (IBA), Karachi) as a pilot run in the country. Although there are many studies on the implementation of ERP systems in business, less research has been reported on the impact of ERP systems in higher education. In Pakistan, some small-scale research has focused on the implementation and efficiency issues of the ERP systems, which suggests that there might be a lack of research in the evaluation of ERP systems in the university environment [4, 5]. The reported research in the contextual areas mainly identified the Critical Success Factors (CSFs) for the effective implementation of ERP systems [1, 6]. The current research study intends to measure the post implementation impact of an ERP system in higher education institutions, focusing on the improvement in the academic and administrative processes [7]. The main reasons that Higher Education Institutions (HEIs) are implementing ERP systems are the needs to replace the legacy manual systems (traditional manual systems), to improve customer service, to increase the transparency in the processes, and to enhance operating efficiency [1, 8]. The current study evaluates the impacts of ERP systems at the end user level where the actual benefits and impacts can be measured. The results of the study can be used by the HEC and Corresponding author: Zahid Hussain Khand Engineering, Technology & Applied Science Research Vol. 10, No. 5, 2020, 6242-6248 6243 www.etasr.com Khand & Kalhoro: Testing and Validating DeLone and MacLean IS Model: ERP System Success in … HEIs to inform further expansion of the ERP systems to other Universities in Pakistan. In particular, this research aims is to explore the impact of ERP implementation in higher education institutions in Pakistan to improve the academic and administrative processes [5]. ERP implementation in HEIs is different from business organizations because they have different environments and conditions. The ERP systems are used by the HEIs to support faculty and students in the fulfillment of a variety of academic needs [5]. To create a high level impact of the research study, the results would be shared with higher education leadership for making decisions regarding the implementation of ERP in different HEIs of Pakistan in the future [9]. II. LITERATURE REVIEW An ERP system is the most widely accepted option for Universities that seek to gain a competitive edge and increase their efficiency and transparency through the integration of all information flowing through the different departments [10, 11]. Despite this perceived positive impact of the ERP system in HEIs, implementation and integration is considered to be a costly and complex solution [12]. This has led to the need to evaluate the system from a stakeholder’s and technical perspective [13]. Most previous research studies on similar topics are related to pre or post implementation issues related to the stakeholder’s perspective or the technical input [14]. Few studies have reported on the CSFs related to the implementation of the ERP system life cycle, or on the academic advantage that the system implementation might foster. There is not a common agreement in which CSFs are most important, but most studies suggest that the issues related to top-management support, technical support, user training, education level of the users and organizational culture are important factors in a successful ERP system implementation [12, 15, 16]. There are a few studies that have reported the success ratio of the ERP systems implementation in higher education and analyze the factors and reasons of failure of such systems. Authors in [17] reported that the failure rate at the implementation stage is 40% to 60%. However different researchers have different definitions of failure. The critical failure factors reported in literature are high turnover of project team, over-reliance on heavy customizations, poor consultant effectiveness, poor IT infrastructure [10, 17, 19], and poor expected Return on Investment (ROI) [18]. The studies reported in the literature tested and validated the ERP success factors using different theoretical Information System (IS) models [12, 15, 16]. However, different theoretical models to measure IS success, have been reported such as: the Technology Acceptance Model (TAM), the Technology- Organization-Environment (TOE) framework, the DeLone and McLean model and the Diffusion of Innovation (DOI). Each IS success model has strengths and weaknesses. The research studies reported that IS success models were used on their own or sometimes in combination with others to measure ERP success in organizations. The literature review did not find any research work that studied the impact of ERPs in HEIs with particular emphasis on the support on academic processes and higher education leadership decision-making. There are a few studies that measure the efficiency of the ERPs in the context of higher education institutions of Pakistan. Authors in [6] proposed a conceptual model based on few suggested changes in the D&M model. The research study also analyzed the success and failure of the ERP systems implemented in eight Universities in Pakistan based on four CSFs. Authors in [20] conducted a similar research study to evaluate the performance of ERP in Saudi universities. The results showed that system’s quality and service quality have positive impact on the stakeholders’ performance. Authors in [21] studied the impact of ERP systems in higher education. The study developed an IS successful model based on the key ideas of TAM, TTF, and D&M models. The findings indicated that system quality, task technology fit, and information quality are the key factors that lead to better end-user performance. Author [7] investigated the success of ERP in Pakistan with end-user perspective and identified CSFs such as top management support, effective project management, clear goals and objectives, selection of ERP systems and data accuracy. Authors in [22] measured the effectiveness of ERP systems in the corporate sector. The research combined the D&M and TAM models and the findings suggested that the performance of the employees can be enhanced through high quality ERP usage. It can be seen that there is a noticeable research gap in this area [20]. A. Research Model and Hypotheses The DeLone and McLean IS success model is the most widely cited model in IS literature [23, 24]. Authors in [25] stated that the model is important in IS success measurement because it provides a classification of all the evaluation measures that have been reported in IS success measurement in the literature. The model identifies potential stakeholders and suggests the way the different constructs interact with each other. The model proposes six dimensions: System Quality, Information Quality, Use, User Satisfaction, Individual Impact and Organizational Impact. The original model is shown in [26] (Figure 1), which depicts that system quality and information quality have direct effect on system usage and user satisfaction. This means that if the users are comfortable with the system functionality and quality of service they are likely to use the system. However, the proposed ERP success model is based on different IS theories in the context of ERP implementation in HEIs. The model is selected after extensive literature review on IS success models [27]. Many empirical studies have been reported regarding IS success measurement based on the D&M model [25, 28]. B. IS Success Variable Categories An IS is defined as a set of components that are integrated and interconnected to collect, process, store, and distribute information to support decision making and supervision [29, 30]. ISs have six dimensions of success, ERP System Quality, ERP Information Quality, ERP Service Quality, ERP Use, ERP User Satisfaction, and ERP Net Benefits [26]. 1) System Quality In the proposed model, System Quality focuses on the ERP’s characteristics System Quality is defined as a set of constructs related to the IS that determine its quality. These Engineering, Technology & Applied Science Research Vol. 10, No. 5, 2020, 6242-6248 6244 www.etasr.com Khand & Kalhoro: Testing and Validating DeLone and MacLean IS Model: ERP System Success in … characteristics include: system reliability, ease to learn, ease of use, productivity, flexibility, and integration with other systems [26, 29, 31, 32]. 2) Information Quality Information Quality is the degree to which information presents the required benefits. The information quality is measured as the output of the IS [30]. These constructs are refined from the literature and include, availability, timelines, relevant, accuracy, appropriateness, concise representation, and interpretability [26, 29, 31, 32]. 3) Service Quality Service Quality is a set of characteristics related to services submitted by the IS to the customer which include service reliability, support, assurance, empathy, and security. Service Quality is considered as the overall support offered by the provider of the IS by ensuring that it can be applied in the proper way [26, 29, 31, 32]. 4) Use Is the extent to which the end-users use the result presented by the ISs. The measures of the Use are defined from the literature and include the degree of system use, responsiveness, adaptability, and effectiveness [26, 29, 31, 32]. 5) User Satisfaction User Satisfaction is the response of the recipient after using the IS [30]. It is associated with attitudes toward IS which include system availability, robustness, task achievement, productivity and efficiency [26, 29, 31, 32]. 6) Net Benefits Net Benefits are the benefits obtained after using the IS. They may be individual or collective. They are a measure for the positive and negative impacts of the IS on all persons and groups effected with IS. Net benefits can be identified with a set of constructs that includes cost saving, expanded markets, incremental additional sale, reduced search cost and time saving [26, 29, 31, 32]. The current study will look into the impact of the ERP systems and their components as suggested above, to provide support in academic processes and management in decision- making. Thus, the following nine hypotheses will be tested in the ERP context: • H1: There is a positive relationship between Information Quality and System Use. • H2: There is a positive relationship between System Quality and System Use. • H3: There is a positive relationship between Service Quality and System Use. • H4: There is a positive relationship between Information Quality and User Satisfaction. • H5: There is a positive relationship between System Quality and User Satisfaction. • H6: There is a positive relationship between Service Quality and User Satisfaction. • H7: There is a positive relationship between System Use and User Satisfaction. • H8: There is a positive relationship between System Use and Perceived Net Benefits. • H9: There is a positive relationship between User Satisfaction and Perceived Net Benefits. Fig. 1. Research framework. III. DATA COLLECTION A survey questionnaire was used to collect the quantitative data from the target population. The targeted population for the collection of data was students, faculty members, and the heads of the academic departments. The majority of the respondents were already ICT literate, so they were easily accessible through email correspondence. The questionnaires were designed using the online tool Survey Monkey and were sent to the respondents through an invitation email. Invitations were sent to 500 respondents (300 students and 200 faculty members). In response to that, 232 students and 141 faculty members filled the questionnaires giving a response rate of 74.6%. After data cleanup, 230 questionnaires were retained for data analysis. IV. DATA ANALYSIS Structural equation modeling (SEM) was used to test the established relationships. For that, this study used the partial least squares (PLS) method by using Smart PLS 3.2.7 software [33]. This method utilizes the bootstrapping processes proposed in [34, 35] to underline the level of significance for loadings and path coefficients for the established relationships. The PLS path modeling method is carried out in two steps (measurement model and structural model). A. Demographic Information of the Respondents The respondent’s demographic information can be seen in Table I. B. Measurement Model Before testing the established relationships, the suitability of the measurement model was assessed based on reliability, convergent validity, and discriminant validity. The reliability was inspected with Composite Reliability (CR) values. Table II indicates that all scores relating to CR range from 0.809 to 0.885, which are higher than the criterion of 0.7, fulfilling the required level [34]. Engineering, Technology & Applied Science Research Vol. 10, No. 5, 2020, 6242-6248 6245 www.etasr.com Khand & Kalhoro: Testing and Validating DeLone and MacLean IS Model: ERP System Success in … TABLE I. DEMOGRAPHIC INFORMATION Variables Category Frequency (%) Gender Male 196 85.22 Female 34 14.78 Age 21 - 25 25 10.87 25 - 30 59 25.65 30 - 40 83 36.09 40 - 45 48 20.87 45 - 50 10 4.35 Above 50 5 2.17 Education Higher secondary (12 years of education) 35 15.22 Undergraduate (16 years of education) 55 23.91 Graduate MS/MPhil (18 years of education) 88 38.26 PhD 52 22.61 University IBA Karachi 26 11.3 Sukkur IBA University 204 88.7 Department Business Administration 98 42.6 Computer Science 38 16.5 Electrical Engineering 35 15.2 Mathematics 15 6.5 Education 23 10.0 Other 21 9.1 Use of ERP system Once a day 37 16.1 Several times a day 90 39.1 Once in a week 39 17.0 Several times in a week 64 27.8 TABLE II. MEASUREMENT MODEL ITEMS Model construct Measurement item Loading CR AVE Information Quality(IQ) IQ1 0.678 0.834 0.46 IQ2 0.749 IQ3 0.688 IQ4 0.756 IQ5 0.668 IQ6 0.499 Net Benefits (NB) NB1 0.635 0.863 0.476 NB2 0.673 NB3 0.728 NB4 0.753 NB5 0.735 NB6 0.696 NB7 0.595 Service Quality (SrQ) SrQ1 0.781 0.859 0.605 SrQ2 0.758 SrQ3 0.825 SrQ4 0.744 System Quality (SQ) SQ1 0.584 0.847 0.483 SQ2 0.766 SQ3 0.775 SQ4 0.689 SQ5 0.691 SQ6 0.644 Use (U) U1 0.850 0.885 0.719 U2 0.863 U3 0.831 User Satisfaction (US) US1 0.585 0.809 0.378 US2 0.663 US3 0.546 US4 0.598 US5 0.647 US6 0.620 US7 0.635 Note: CR = Composite Reliability, AVE = Average Variance Extracted Moreover, using item loadings tested convergent validity and the AVE criteria. The results shown in Table II imply that the loading values of all the items are higher than the 0.5 threshold, indicating that the study has achieved scale validity. The values of AVE of two variables (Service Quality and Use) exceeded the recommended value (0.5) suggesting that convergent validity is at acceptance level. However, AVE values of three constructs (System Quality, Information Quality and Net Benefit) ranged from 0.46 to 0.483, which are near the threshold value of 0.5, whereas the AVE value of one construct (User Satisfaction) is 0.378. Table II shows the result of discriminant validity. It was examined using the square root of the average variance extracted criteria suggested in [36], according to which, the square root of the AVE should be greater than cross-correlations within construct and correlation with other constructs in the model. All the values of square root of AVE of all constructs except one (Net Benefit) shown in Table III surpassed the inter-construct correlations coefficient, suggesting that discriminant validity is acceptable. TABLE III. DISCRIMINANT VALIDITY Variables IQ NB SrQ SQ U US Information Quality 0.67 Net Benefit 0.55 0.69 Service Quality 0.72 0.64 0.77 System Quality 0.72 0.59 0.66 0.69 Use 0.56 0.60 0.61 0.56 0.84 User Satisfaction 0.60 0.78 0.63 0.56 0.50 0.61 Note: Values in the diagonal represent the square root of the average variance extracted. C. Structural Equation Model Fig. 2. PLS output. With the achievement of adequate validity and reliability for the measurement model of the study, the proposed structural model with established relationships was estimated by applying 500 iterations of the bootstrapping technique. Table IV and Figure 2 show the results of the hypotheses testing. The results indicate that Information Quality had not significant impact on use (H1 rejected, �=0.141, � � 0.1� but had significant influence on User Satisfaction (H4 supported, �=0.22, � 0.05). Service Quality exerted significant impact on both Use and User Satisfaction at � 0.01. Therefore, H3 and H6 were accepted (�=0.36 and �=0.321 respectively). Engineering, Technology & Applied Science Research Vol. 10, No. 5, 2020, 6242-6248 6246 www.etasr.com Khand & Kalhoro: Testing and Validating DeLone and MacLean IS Model: ERP System Success in … Moreover, System Quality exerted significant influence on use (�=0.224, � 0.01�, but had not significant effect on User Satisfaction (�=0.12, � � 0.1�, thus H5 was rejected and H2 was supported. Additionally, Use had exerted significant impact on both Net Benefit and User Satisfaction at � 0.01 and � 0.1 respectively. Hence, H8 and H7 were supported (�=0.282 and �=0.12, respectively). TABLE IV. HYPOTHESES TESTING RESULTS Estimate Std error t-value p-value IQ ���� U 0.14 0.09 1.559 0.120 IQ ���� US 0.22 0.10 2.154 0.032 SrQ ���� U 0.36 0.07 5.166 0.000 SrQ ���� US 0.32 0.074 4.358 0.000 SQ ���� U 0.22 0.074 3.026 0.003 SQ ���� US 0.12 0.081 1.478 0.140 U ���� NB 0.28 0.045 6.194 0.000 U ���� US 0.12 0.07 1.72 0.086 US ���� NB 0.64 0.04 15.62 0.00 User Satisfaction had significant influence on Net Benefits ( � =0.642, � 0.01� , indicating that H9 was supported. Overall, the results show that Use and User Satisfaction accounted for 67.5% variance in Net Benefit. Information Quality, System Quality, and Service Quality accounted for 42.6% variance in Use and Information Quality, System Quality, Service Quality and Use accounted for 46.5% variance in User Satisfaction. Table V indicates that Information Quality, System Quality, Service Quality, Use, and User Satisfaction show significant total effects on the Net Benefits being all significant at P<0.01. Furthermore, the total effect of User Satisfaction on Net Benefits is stronger than Use. Likewise, among all quality related constructs, Service Quality shows strongest total effect (0.335) on Net Benefits. TABLE V. TOTAL EFFECTS Estimate Std error t-value p-value IQ ���� NB 0.19 0.07 2.66 0.00 SrQ ���� NB 0.33 0.05 6.24 0.00 SQ ���� NB 0.15 0.05 2.77 0.00 U ���� NB 0.35 0.06 5.84 0.00 US ���� NB 0.64 0.04 15.62 0.00 V. CONCLUSION AND FUTURE WORK The current study presents and validates a model of ERP system success based on an updated D&M IS success model [25] with six success measures that are Information Quality, System Quality, Service Quality, Use, User Satisfaction and Net Benefits. Except for the link between IS and Use and a link between System Quality and User Satisfaction all other hypothesized relationships between the six ERP success variables were significantly or marginally supported by the data [29]. The study recommends more research in this area and finding more variables that have impact on System Use and User Satisfaction. The current study has a few limitations that could be addressed in the future. First, measuring the impact of ERP systems in higher education with D&M IS success model is a new research area. The IS success model can be re- specified with more variables related with the ERP context. Second, the empirical measurement sample size and the number of universities may be increased to make the findings more generalized. Third, future research may be done with longitudinal evidence that could enhance our understanding to replicate the current study with respect to time and long-term usage. APPENDIX A- QUESTIONNAIRE Section-A Demographic Information 1. Are you using the Campus Management Solution (CMS) System at the University? ( ) Yes ( ) No 2. Name of the University: ( ) IBA, Karachi ( ) Sukkur IBA University. 3. Department: ( ) Business Administration ( ) Computer Science ( ) Electrical Engineering ( ) Mathematics ( ) Education ( ) Other 4. How long have you been using Enterprise Resource Planning (ERP) in your University? ( ) months ( ) years 5. Gender: ( ) Male ( ) Female 6. Age: ( ) Less than 21 ( ) 21 and below 25 years ( ) 25 and below 30 years ( ) 30 and below 40 years ( ) 40 and below 45 years ( ) 45 and below 50 years ( ) 50 years and above 7. Education: Please specify the highest qualification you have attained: ( ) Higher Secondary School Certificate (12 years of education) ( ) Undergraduate (16 years of education) ( ) Graduate MS/MPhil (18 years of education) ( ) Doctoral PhD 8. How often do you use the system: ( ) Once a day ( ) Several times a day ( ) Once in a week ( ) Several times in a week Section-B Please rate to what extent you agree or disagree with each statement below by CLICKING the most appropriate number of the scales provided. Please click ONLY ONE option for each statement given below. System Quality Information quality is measured in terms of ease-of-use, functionality, reliability, flexibility, data quality, portability, etc. Engineering, Technology & Applied Science Research Vol. 10, No. 5, 2020, 6242-6248 6247 www.etasr.com Khand & Kalhoro: Testing and Validating DeLone and MacLean IS Model: ERP System Success in … 1. The Campus Management Solution (CMS) System is easy to use. 2. The CMS is effective in providing instant access to information required. 3. The CMS System streamlines the best academic practices in the University. 4. The CMS supports me in completing my academic related tasks efficiently. 5. The CMS is effective in providing accurate information about student attendance and grades. 6. The CMS provides complete attendance automation by effectively sending SMS and Email alerts to students. Information Quality Information quality is measured in terms of accuracy, timeliness, completeness, relevance, and consistency. 7. The CMS system effectively provides me with relevant information. 8. The information required is easily accessible in the CMS system. 9. The CMS system is easy to learn. 10. The CMS user interface can easily be configured to my personal requirements. 11. I think that my data in the CMS is safe from unauthorized access. 12. The CMS effectively generates text and email alerts if someone tries to enter my account unauthorized. Service Quality Service quality is used to measures reliability, responsiveness, assurance, and empathy. 12. The CMS system provides reliable information. 13. The information in CMS is easily searchable. 14. The CMS provides a smooth flow of information regarding the academic calendar, assignments and announcements of results. 15. The CMS provides access to grades, attendance, fee status, exam schedule and announcements of results related to students. Impact of ERP 16. 17. The CMS improves the academic processes in the University. 18. The CMS system effectively combines the student data from different departments of the University. 17. 19. The CMS provides me single login ID that improves the information integration between different systems of the University. User Satisfaction 20. The CMS helps in sharing the scholarship related data of students with potential donors. 21. The CMS helps me in getting updated information about my attendance and grades on regular basis. 22. The CMS gives students a chance to provide course evaluations in each semester. 23. The feedback given by students in the course evaluations has helped to improve classroom teaching. 24. The CMS/ERP self-service automates many processes related with students (visibility of semester results, course registration and financial matters). 25. I received adequate CMS related trainings during my studies to perform my academic tasks on system effectively. 26. It is easy to get required support from IT support team whenever required. Net Benefits 27. It is useful that the CMS manages classroom information and analytical reports of the student grades and attendance. 28. The CMS helps me to improve my academic performance by receiving SMS alerts on attendance and grades. 29. The CMS provides a chance to students to provide feedback on the courses offered in a semester. 30. The CMS improves the internal communication within the university (students, faculty and administration). 31. The CMS effectively provides chance to register or drop in different courses in a semester through self-service module as compared from what we had before. 32. The CMS provides a chance to students to provide feedback at the end of a semester about their overall experience regarding classroom teaching/learning for courses offered in a semester. 33. The CMS provides grades related information, which may help students to monitor their progress during the semester. 34. Are there any additional comments about your experiences with CMS/ERP? REFERENCES [1] A. Abugabah and L. Sanzogni, “Enterprise Resource Planning (ERP) System in Higher Education: A Literature Review and Implications,” International Journal of Computer and Systems Engineering, vol. 4, no. 11, pp. 2120–2124, Nov. 2010. [2] O. O. Olugbara, B. M. Kalema, and R. M. Kekwaletswe, “Identifying critical success factors: the case of ERP systems in higher education,” The African Journal of Information Systems, vol. 6, no. 3, pp. 65–84, Jul. 2014. [3] A. S. Shatat, “The Impact of ERP System on Academic Performance: A Case Study Approach,” Journal of Information & Knowledge Management, vol. 18, no. 02, May 2019, doi: 10.1142/ S0219649219500187, Art. No. 1950018. [4] M. Y. Jamil and R. Qayyum, “Enterprise Resource Planning (ERP) Implementation in Pakistani Enterprises: Critical Success Factors and Challenges,” Journal of Management and Research, vol. 2, no. 2, pp. 1– 35, 2015, doi: 10.29145/jmr/22/0202004. [5] Z. Ahmer, “Usage of Enterprise Resource Planning Systems in Higher Education Institutions in Pakistan,” Ph.D. dissertation, University of Hertfordshire, United Kingdom, 2017. [6] S. Nizamani, K. Khoumbati, I. Ismaili, and S. Nizamani, “A Conceptual Framework for ERP Evaluation in Universities of Pakistan,” Sindh University Research Journal (Science Series), vol. 45, no. 3, pp. 596– 607, 2013. [7] M. Sadiq, Investigating the success of ERP systems in Pakistan : end- users’ perspective. Jyväskylä, Finland: University of Jyväskylä, 2016. [8] M. S. Abdel-Haq, H. Chatti, and E. Asfoura, “Investigating the Success and the Advantages of Using ERP System in KSA Context,” Engineering, Technology & Applied Science Research, vol. 8, no. 6, pp. 3631–3639, Dec. 2018. [9] F. Schwade and P. Schubert, “The ERP Challenge: Developing an Integrated Platform and Course Concept for Teaching ERP Skills in Universities,” International Journal of Human Capital and Information Technology Professionals (IJHCITP), vol. 9, no. 1, Jan. 2018, doi: 10.4018/IJHCITP.2018010104. [10] G. Seo, “Challenges in implementing enterprise resource planning (ERP) system in large organizations : similarities and differences between corporate and university environment,” M.S. thesis, Massachusetts Institute of Technology, 2013. [11] N. Karia and M. Soliman, “Higher Education Competitive Advantage: Enterprise Resource Planning Systems,” IRACST- International Journal of Research in Management & Technology (IJRMT), vol. 5, no. 5, pp. 380–384, Oct. 2015. Engineering, Technology & Applied Science Research Vol. 10, No. 5, 2020, 6242-6248 6248 www.etasr.com Khand & Kalhoro: Testing and Validating DeLone and MacLean IS Model: ERP System Success in … [12] M. Soliman and N. Karia, “Antecedents for the Success of the Adoption of Organizational ERP Among Higher Education Institutions and Competitive Advantage in Egypt,” Engineering, Technology & Applied Science Research, vol. 7, no. 3, pp. 1719–1724, Jun. 2017. [13] S. Rani, “A Review of ERP Implementation in Higher Education Institutions,” International Journal of Advanced Research in Computer Science and Software Engineering, vol. 6, no. 6, pp. 542–545, Jun. 2016. [14] A. Rabaai, W. Bandara, and G. Gable, “ERP systems in the higher education sector : a descriptive case study,” in Proceedings of the Twentieth Australasian Conference on Information Systems, M. Davern and H. Scheepers, Eds. Monash University, 2009, pp. 456–470. [15] Liang Zhang, M. K. O. Lee, Zhe Zhang, and P. Banerjee, “Critical success factors of enterprise resource planning systems implementation success in China,” presented at the 36th Annual Hawaii International Conference on System Sciences, 2003, Big Island, HI, USA, Jan. 2003, doi: 10.1109/HICSS.2003.1174613. [16] S. Reilly, J. Healy, T. Murphy, and R. O’Dubhghaill, “A Continuous Improvement Journey in the Higher Education Sector: A Case Study of a University in Ireland,” in Proceedings of the 4th International Conference on Lean Six Sigma for Higher Education, May 2017, pp. 7– 15, doi: 10.5703/1288284316371. [17] A. Wong, H. Scarbrough, P. Chau, and R. Davison, “Critical Failure Factors in ERP Implementation.,” presented at the Pacific Asia Conference on Information Systems, PACIS 2005, Bangkok, Thailand, Jul. 2005. [18] C. A. Ptak and E. Schragenheim, ERP: Tools, Techniques, and Applications for Integrating the Supply Chain, Second Edition, 2 edition. Boca Raton: CRC Press, 2003. [19] P. Garg and A. Garg, “An empirical study on critical failure factors for enterprise resource planning implementation in Indian retail sector,” Business Process Management Journal, vol. 19, no. 3, pp. 496–514, Jan. 2013, doi: 10.1108/14637151311319923. [20] M. Althonayan and A. Papazafeiropoulou, “Evaluating the Performance on ERP Systems in King Saud University (KSU): A Stakeholders’ Perspective,” in 2013 46th Hawaii International Conference on System Sciences, Jan. 2013, pp. 4074–4083, doi: 10.1109/HICSS.2013.210. [21] A. Abugabah, L. Sanzogni, and O. Alfarraj, “Evaluating the impact of ERP systems in higher education,” The International Journal of Information and Learning Technology, vol. 32, no. 1, pp. 45–64, Jan. 2015, doi: 10.1108/IJILT-10-2013-0058. [22] R. N. Lodhi, M. Abdullah, and A. Shahzad, “An empirical investigation of the effectiveness of ERP quality: Evidence from corporate sector of Pakistan,” Journal of Quality & Technology Management, vol. 12, no. 2, pp. 71–88, Dec. 2016. [23] S. Petter, W. DeLone, and E. McLean, “Measuring information systems success: models, dimensions, measures, and interrelationships,” European Journal of Information Systems, vol. 17, no. 3, pp. 236–263, Jun. 2008, doi: 10.1057/ejis.2008.15. [24] A. Daghouri, K. Mansouri, and M. Qbadou, “Enhanced Model For Evaluating Information System Success: Determining Critical Criteria,” Engineering, Technology & Applied Science Research, vol. 8, no. 4, pp. 3194–3198, Aug. 2018. [25] Y.-S. Wang and Y.-W. Liao, “Assessing eGovernment systems success: A validation of the DeLone and McLean model of information systems success,” Government Information Quarterly, vol. 25, no. 4, pp. 717– 733, Oct. 2008, doi: 10.1016/j.giq.2007.06.002. [26] W. H. Delone and E. R. McLean, “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update: Journal of Management Information Systems: Vol 19, No 4,” Journal of Management Information Systems, vol. 19, no. 4, pp. 9-30, doi: 10.1080/ 07421222.2003.11045748. [27] A. Manchanda and S. Mukherjee, “A review of information system success models,” International Journal of Innovative Research in Technology & Science (IJIRTS), vol. 1, no. 3, pp. 15–18, Jul. 2013. [28] J. Iivari, “An empirical test of the DeLone-McLean model of information system success,” ACM SIGMIS Database: the DATABASE for Advances in Information Systems, vol. 36, no. 2, pp. 8–27, Jun. 2005, doi: 10.1145/1066149.1066152. [29] I. Romi, “Testing Delone and Mclean’s Model in Financial Institutions,” American Academic & Scholarly Research Journal, vol. 5, no. 3, pp. 121–129, May 2013. [30] H. Irawan and I. Syah, “Evaluation of implementation of enterprise resource planning information system with DeLone and McLean model approach,” in 2017 5th International Conference on Information and Communication Technology (ICoIC7), May 2017, pp. 1–7, doi: 10.1109/ICoICT.2017.8074721. [31] K. S. Wei, A. C. Y. Loong, Y. M. Leong, and K. B. Ooi, “Measuring ERP System Success: A Respecification of the Delone and Mclean’s is Success Model,” in TechRepublic, 2009, Accessed: Aug. 23, 2020. [Online]. Available: https://www.techrepublic.com/resource-library/ whitepapers/measuring-erp-system-success-a-respecification-of-the- delone-and-mclean-s-is-success-model/. [32] P. A. E. Serumaga-Zake, “The role of user satisfaction in implementing a Business Intelligence System,” SA Journal of Information Management, vol. 19, no. 1, p. 8, Feb. 2017, doi: 10.4102/sajim .v19i1.736. [33] C. M. Ringle, S. Wende, and J. M. Becker, SmartPLS 3. Boenningstedt, Germany: SmartPLS GmbH, 2015. [34] J. F. Hair, M. Sarstedt, C. M. Ringle, and J. A. Mena, “An assessment of the use of partial least squares structural equation modeling in marketing research,” Journal of the Academy of Marketing Science, vol. 40, no. 3, pp. 414–433, May 2012, doi: 10.1007/s11747-011-0261-6. [35] J. Hulland, “Use of partial least squares (PLS) in strategic management research: a review of four recent studies,” Strategic Management Journal, vol. 20, no. 2, pp. 195–204, 1999, doi: 10.1002/(SICI)1097- 0266(199902)20:2<195::AID-SMJ13>3.0.CO;2-7. [36] C. Fornell and D. F. Larcker, “Evaluating Structural Equation Models with Unobservable Variables and Measurement Error,” Journal of Marketing Research, vol. 18, no. 1, pp. 39–50, 1981, doi: 10.2307/3151312.