Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 119 Journal of Applied Economics and Business Studies (JAEBS) Journal homepage: https://pepri.edu.pk/jaebs ISSN (Print): 2523-2614 ISSN (Online) 2663-693X Enterprise Resource Planning Systems and User Performance in Higher Education Institutions of Pakistan Abrar Ullah 1,2*, Rohaizat Bin Baharun1, Muhammad Yasir3 & Khalil MD Nor 1 1 Azman Hashim International Business School, Universiti Teknologi Malaysia, Johor, 81310, Malaysia 2 Department of Management Sciences, University of Swabi, Swabi, Pakistan 3 Department of Management Sciences, Bacha Khan University, Charsadda, Pakistan ABSTRACT The study is designed to assess the impact of Enterprise Resource Planning (ERP) systems on the performance perceived by users in Higher Education Institutions (HEIs) of Pakistan. This study sought to evaluate the effect of ERPs quality factors including Information Quality (IQ), System Quality (SQ) and Service Quality (SRQ) on User Performance (UP) towards system usage through the role of Perceived Usefulness (PU), Perceived Ease of Use (PEOU) and User Satisfaction (US). Consequently, a framework is proposed by integration of DeLone & McLean (D&M) Information Systems (IS) success model and Technology Acceptance Model (TAM) to address the research questions. The study used quantitative research methodology and data were collected from 317 employees from eight universities in Pakistan. Structural Equation Modelling (SEM) was performed using SmartPLS. The results indicated that SQ, IQ and SRQ has direct and positive effect on UP, PU, PEOU and US. Additionally, PU, PEOU and US are found to have influence on UP. Theoretically, the study contributes by integrating factors from D&M IS success model and TAM to investigate the effect of ERP systems on UP through PU, PEOU and US. Practically, the study implied that practitioners needs to put efforts to provide a system which users perceive as useful and free of efforts. Keywords Information Systems, ERP Systems, User Performance, Management Information Systems, Higher Education JEL Classification C32, C38, C52, C58 1. Introduction The term Enterprise Resource Planning (ERP) was introduced by Gartner Group in 1990s (Arif et al., 2004), comprising of “computer software systems that integrate all related processes within enterprise and provides users with services to manage all functions” (Swartz et al., 2001). ERP systems implementation brings great advantages (Ullah et al., 2018; Umar et al., 2016), but due to the uncertainties of technological complexities, the completion of these projects remains a challenge (Xu et al., 2010). Millions of dollars are invested on ERP systems (Beheshti et al., 2010), and despite of enormous growth, it is claimed that these systems failed * abrar.ullah@uoswabi.edu.pk Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 120 with a higher rate of about to be 60 -90 percent (Ahmad et al., 2014; AlShamlan et al., 2011; Gill et al., 2020). Similarly, it is reported that the organization’s expenditure on ERP systems implementation is about $6.1 million, of which 58% are cost overrun, 65% experienced schedule overrun, and in the post implementation stage 53% achieved less than 50-percent of anticipated measurable benefits (Solutions, 2015). Thus, making this a thorny problem deserving further exploration that how to achieve related outcome from implemented systems (Sun et al., 2015). In the IS related research, assessing its impact is a crucial concern (Petter et al., 2008) and it is repeatedly reported as the main problem by organizational administrators (Gable et al., 2008). Practitioners as well as researchers yet to find answers for how to measure IS successfully (Rabaa'i et al., 2009). In this quest, several models and theories have emerged to measure and recognize technology and specifically IS such as, the Social Cognitive Theory (Bandura, 1986), D&M IS success model (DeLone et al., 1992, 2003), Theory of Reasoned Action (TRA), Diffusion of Innovation (DOI) theory (Rogers, 1995), Theory of Planned Behavior, the Unified Theory of Acceptance and Use of Technology (UTAUT) (Venkatesh et al., 2003), TAM (Davis, 1989). However, with the continues innovation in technologies, the focus is shifted towards the outcome in terms of performance while evaluating system success (A. H. Aldholay et al., 2018; Isaac et al., 2017). Within the ERP environment, previous studies mentioned performance evaluation of these systems and only some of the reported articles discussed its impact on the productivity and performance (Shen et al., 2016). Reviewing the literature related to ERP system performance shows that majority of prior studies measure the ERP systems in terms of performance at organizational level rather than individual and the outcome variables discussed are product quality (Banker et al., 2006), benefits (Zhu et al., 2010), financial performance (Ayman et al., 2015), organizational service enhancement (Gorla et al., 2010), market value (Ranganathan et al., 2006), process efficiency (Chou et al., 2008), shareholder return (Galy et al., 2014), competitive advantage (Ram et al., 2014), organisational benefits (Almahamid et al., 2015). Thus, providing enough evidence to investigate from users perspective at individual level rather organizational level, that is investigating the actual impacts of ERP systems on users (Hsu et al., 2015), as users itself are consumers while receiving services from IT department (Alsaleh et al., 2016) and user performance studies are given less attention (Ullah, Baharun, Nor, Siddique, & Sami, 2018). Although, ERP systems have implemented in large manufacturing organizations, but research in higher education is still limited (Albarghouthi et al., 2020), especially in Pakistan (Ahmer et al., 2018; Nizamani et al., 2014). In summary, the preceding background presents the motivation for the research, and it shows that studies tested these models in different areas with different stakeholders and systems. Some studies adopted these models partially and tested specific constructs, while others tried to extend the models by adding additional constructs. Keeping in view the above, Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 121 D & M model and TAM integration is another route to evaluate antecedents that determine the ERP systems success in higher education of Pakistan. Accordingly, objective of the study to propose a model based on D & M IS success model representing quality factors and TAM to examine whether quality factors as antecedents to acceptance and user satisfaction effect user performance while using ERP system. 2. Literature review and development of hypotheses 2.1 System quality In related literature to IS, quality is somewhat “ill-defined” (Nelson et al., 2005) and debate exists among authors that the quality factors are either self-defined or derived empirically (Ullah, Baharun, Nor, Siddique, & Bhatti, 2018). Rai et al. (2002) defined, System Quality as “the degree to which a system is user friendly”. System quality is often found relevant construct and always found in support, while evaluating the matters of IS (Urbach et al., 2010) and is studied widely as explanatory construct in ERP research (Gorla et al., 2010; Rajan et al., 2015; Sternad et al., 2013). In studies, positive impact of SQ on perceived usefulness has been found. Among them Lin (2010) found the influence of SQ on PU and the study also indicated that SQ can increase the belief about ERP system usefulness. Zhou (2014) investigated system quality influence on perceived usefulness to measure trust in mobile payment and found the effect significant. This relationship is also found positive in other related studies of the ERP system environment (Abugabah et al., 2015; Ali et al., 2013; Cheng, 2019; Gorla et al., 2014; Lin, 2010; Tseng et al., 2018; Yang et al., 2017; Zhou, 2011). In their study on ERP satisfaction and user adoption Costa et al. (2016) found that SQ have positive effect on perceived ease of use. Other related studies also tested and confirmed the effect of SQ on PEOU (Abugabah et al., 2015; Ali et al., 2013; Lin, 2010; Rana et al., 2015; Yang et al., 2017). Satisfied ERP employees may be more efficient, when system usage is mandatory (Hsu et al., 2015) as US refers to “the degree to which users believe that the IS meets their requirements” (Delone et al., 2003). According to Tsai et al. (2012) SQ is the key antecedent of user satisfaction in ERP environment. Therefore, we expect that the SQ has a significant influence on US. Prior studies backed this claim and showed significant effect of SQ on US such as (Cheng, 2019; Costa et al., 2016; Landrum et al., 2010; Lin, 2010; Masrek et al., 2016; Noorman Masrek et al., 2010; Rana et al., 2015; Shahibi et al., 2016; Tam et al., 2016). Based on these pieces of evidence, it is postulated that: H1: System quality positively influence perceived usefulness H2: System quality positively influence perceived ease of use H3: System quality positively influence user satisfaction Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 122 2.2 Information quality Information Quality (IQ) is “the degree to which information generated possess content, accuracy, and format” (Rai et al., 2002). Higher information quality enhances performance of users with system usage, reinforcing the perceptions of usefulness (Chen et al., 2015). Researchers supported the effect of information quality on perceived usefulness like (Abugabah et al., 2015); Floropoulos et al. (2010). In similar environment Zhou (2011) found the positive effect of IQ on PU in the study of mobile web sites adoption. Other researchers (Alfarraj et al., 2017; Chen et al., 2015; Cheng, 2019; Lin, 2010; Tseng et al., 2018; E. S.-T. Wang, 2016; Zhou, 2014) also confirmed this effect. The effect of information quality on perceived ease of use was found by the study of Ali et al. (2013) while evaluating IS impact on user performance. Their result confirmed that user performance is improved when system is useful and easy to use. This relationship is also explored in studies of (Abugabah et al., 2015; Rana et al., 2015; Zhou, 2011) in different settings. Information quality plays a vital role to satisfy users to achieve goals and it depends on the purpose of the users (Y. S. Wang, 2008). Mohammadi (2015a) tested the influence of IQ on user satisfaction to investigate user’s perceptions about e-learning system in universities and found this relationship significant. Moreover, previous researchers found that IQ is a key predictor of user satisfaction (Chen et al., 2015; Cheng, 2019; Floropoulos et al., 2010; Hsu et al., 2015; Landrum et al., 2010; Lin, 2010; Masrek et al., 2016; Mohammadi, 2015b; Noorman Masrek et al., 2010; Shahibi et al., 2016; Tam et al., 2016). These studies showed that satisfaction of users with the system depends on higher quality of information. Hence, it is assumed that: H4: Information quality positively influence perceived usefulness H5: Information quality positively influence perceived ease of use H6: information quality positively influence user satisfaction 2.3 Service quality Petter et al. (2009) described service quality as “the quality of support received by users when interacting with the system”. SRQ is treated as peripheral to SQ and IQ, but recent research claimed that it can be seen as substantial construct due to the improved development in the service role of IS (Mohammadi, 2015a). In previous literature, the influence of service quality on perceived usefulness is confirmed as positive (Ahn et al., 2007; Chen et al., 2015; Floropoulos et al., 2010; Landrum et al., 2010; Yang et al., 2017). In a study by Zhou (2011), it is found that SRQ is a predictor to perceived ease of use and this relationship is also confirmed by Ahn et al. (2007). The relationship of service quality with user satisfaction came from Delone et al. (2003). The same relationship was tested by Masrek et al. (2016) to investigate the determinant of US. Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 123 In ERP related studies, several authors (Alshibly, 2014; Hsu et al., 2015; Landrum et al., 2010; Mohammadi, 2015a; Noorman Masrek et al., 2010; Rana et al., 2015; Shahibi et al., 2016; Tam et al., 2016) confirmed the effect of SRQ on US. Thus, SRQ is assumed to have the same effect: H7: Service quality positively influence perceived usefulness H8: Service quality positively influence perceived ease of use H9: Service quality positively influence user satisfaction 2.4 Relationship between PU, PEOU, US to user performance The constructs perceived usefulness and perceived ease of use came from TAM and this part represents the acceptance of technology. PU and PEOU of TAM are important elements to decide the acceptance of technology of the system (Hsieh et al., 2013). The success of the system also relies on satisfaction level towards the system (Y.-S. Wang et al., 2008). Several studies supported these relationships, for example, Abugabah et al. (2015) confirmed that PU influences the performance of users. The same relationship have been found positive and significant in studies by (Ali et al., 2013; Chen et al., 2015). Similarly, the effect of PEOU on user performance in an ERP environment was also found in studies (Abugabah et al., 2015; Ali et al., 2013). Lastly, the influence of user satisfaction on the UP is confirmed positive and significant in previous studies (Chen et al., 2015; Hsu et al., 2015; Noorman Masrek et al., 2010; Tam et al., 2016). Based on the preceding paragraph, the same outcome is expected. Hence the following is postulated. H10: Perceived usefulness positively influence user performance H11: Perceived ease of use positively influence user performance H12: User satisfaction positively influence user performance 2.5 User performance Scholars used Organizational impact (Gorla et al., 2014), individual impact (Ifinedo et al., 2010), system usage (Lin, 2010), satisfaction (Floropoulos et al., 2010; Landrum et al., 2010), organizational performance (Choi et al., 2013), perceived net benefits (Chen et al., 2015) as dependable variable to measure ERP systems. However, with growth in technology, the focus is diverted to the outcome as performance to measure the success (Abugabah et al., 2015; A. Aldholay et al., 2018; Montesdioca et al., 2015). User performance is referred to the outcome of doing a set of tasks (Alfarraj et al., 2017; Ali et al., 2013; Ullah et al., 2018). In this study, user performance is measured with questions related to efficiency and effectiveness. According to Abugabah et al. (2015) efficiency is “the extent to which the method of performance minimizes the efforts that are required to perform the task”, and the effectiveness is “the degree of the objective accomplishment showing how well a set of results could be accomplished”. In this study context, users are individuals who use ERP system for daily job, have some Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 124 knowledge about the functionality of the system, and are familiar with other users (Liu et al., 2011). 3 Research methods 3.1 Overview of proposed framework In the current study, the constructs and the hypothesized relationships among them are derived from the literature as preceding sections. The framework is shown in Figure 1, where it is mentioned that the ERP quality factors (System quality, Information quality, Service quality) influence perceived usefulness, perceived ease of use and user satisfaction, which further predict user performance. The quality factors are derived from Delone et al. (2003), perceived usefulness and perceived ease of use from Davis (1989) and user performance is taken from Abugabah et al. (2015). The proposed framework set to test 12 hypotheses. Figure 1: Study Framework 3.2 Development of instrument In line with existing literature, a total of 51 items questionnaire was developed to capture the constructs. A 5-point Likert scale is employed for each item of the constructs, with 1 as strongly disagree, 5 strongly agree, and 3 as not applicable (NA). All the items in the questionnaire are adapted from related studies as shown in Table 1 and Appendix B. Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 125 Table 1: Construct measurement 4 Data collection As discussed, the instruments for this study are adapted from established sources but pretesting is still needed to ensure that each question fits well with a different set of respondents (Kumar et al., 2013). The process of pretesting was conducted by a panel comprising of renowned academicians in the field. Each questionnaire item was examined, and cases of ambiguous wording are rectified and rephrased. This process fine-tunes the research instrument based on their feedback and recommendations such as tangibility dimension of service quality was removed as most of the questions remained similar to system quality. Hereafter, the data was collected through google form. The questionnaire was sent by email to all employees in 8 universities in Pakistan. The respondents are ERP system users with the name of Campus Management Solutions (CMS) system. In total, we collected 356 responses from 8 universities. After the completion of initial data screening including missing values and outlier a total of 317 usable sample were retained. The respondents’ demography is presented in Table 2. Table 2: Respondents demographics Demographic Feature Frequency Percentage Gender Male 247 77.9 Female 70 22.1 Age Above 21 and below 25 years 18 5.7 Above 25 and below 30 years 38 12.0 Above 30 and below 35 years 107 33.8 Above 35 and below 40 years 61 19.2 40 and above 93 29.3 Education Higher Secondary School Certificate (HSSC) 3 0.9 Graduation (14 years) 10 3.2 Master (16 years) 45 14.2 MPhil/MS (18 years) 131 41.3 Construct Items Source System Quality SQ1 – SQ9 (Hsu et al., 2015); (Gable et al., 2008); (Nelson et al., 2005) Information Quality IQ1 – IQ6 (Hsu et al., 2015); (Gable et al., 2008); (Nelson et al., 2005) Service Quality Reliability SRQ1 – SRQ4 (Hsu et al., 2015); (Pitt et al., 1995); (Parasuraman et al., 1988) Responsiveness SRQ5 – SRQ7 Assurance SRQ8 – SRQ11 Empathy SRQ12 – SRQ15 Perceived Usefulness PU1 – PU4 (Abugabah et al., 2015) Perceived Ease of Use PEOU1 – PEOU4 (Abugabah et al., 2015) User Satisfaction US1 – US3 (Lin, 2010) User Performance UP1 – UP10 (Abugabah et al., 2015) Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 126 Doctorate 128 40.4 Experience Less than 3 years 61 19.2 3 to 6 years 78 24.6 7 to 10 years 78 24.6 More than 10 years 100 31.5 Usage Once a week 74 23.3 Once a day 69 21.8 Several times a day 51 16.1 Regular use, many times a day 123 38.8 5 Data analysis and results Structure Equation Model (SEM) technique was used for hypotheses testing. Two-stage approach was followed as recommended by Hair et al. (2017) to examine the measurement model and structural model using SmartPLS 3.0 software (C. Ringle et al., 2015; C. M. Ringle et al., 2020). 5.1 Measurement model assessment The measurement model stage determines the reliability and validity of the constructs. This includes convergent, discriminant validity and reliability. Convergent Validity is “the degree to which a measure correlates positively with alternative measures of the same construct” (Hair et al., 2017). For factors loading the recommended value is ≥ 0.708 but loading greater than 0.6 or 0.5 is adequate (Ramayah et al., 2018). On the basis of analysis 7 items found to be short of the required level are deleted. The values of composite reliability (CR) shown in Table 3 were used for construct reliability. As shown CR ranging between 0.796 and 0.929 are over the recommended value of 0.70 by Gefen et al. (2000); Kline (2010). Average Variance Extracted (AVE) criterion was employed to determine convergent validity. AVE ranging between 0.602 and 0.711, meeting the recommended threshold above 0.50 (Hair et al., 2010). Table 3: Reliability and loading Construct Item Loading (>0.5) CR AVE System quality (SQ) SQ2 0.686 0.896 0.612 SQ3 0.777 SQ4 0.791 SQ5 0.781 SQ6 0.650 SQ8 0.727 SQ9 0.785 Information quality (IQ) IQ1 0.776 0.929 0.685 IQ2 0.819 IQ3 0.849 IQ4 0.874 IQ5 0.851 IQ6 0.794 Service quality (SRQ) Reliability (REL) SRQ1 0.662 0.796 0.610 Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 127 Discriminant validity is “the extent to which a construct differs from other constructs in the research model, that is the construct measures what is intended to measure” (Hair et al., 2017). Discriminant validity was determined using three criteria namely Fornell and Lacker’s criterion (Fornell et al., 1981), items cross loading and recently developed Heterotrait-Monotrait Ratio (HTMT) (Henseler et al., 2015). Fornell-Larcker’s criterion is the comparison of square root of AVE values with construct correlations (Hair et al., 2017), in other words, the construct share more variance with its own block of items compare to other construct. Table 4 presents discriminant validity using this criterion is established as the values of each construct’s square root (values presented in bold) are larger than the correlation value between construct. SRQ2 0.750 SRQ3 0.837 Responsive (RESP) SRQ5 0.813 0.872 0.694 SRQ6 0.864 SRQ7 0.821 Assurance (ASSU) SRQ9 0.693 0.810 0.602 SRQ10 0.802 SRQ11 0.802 Empathy (EMP) SRQ12 0.740 0.832 0.624 SRQ13 0.779 SRQ14 0.847 Perceived usefulness (PU) PU1 0.813 0.908 0.711 PU2 0.841 PU3 0.880 PU4 0.836 Perceived ease of use (PEOU) PEOU1 0.686 0.838 0.620 PEOU2 0.621 PEOU3 0.860 PEOU4 0.820 User satisfaction (US) US1 0.843 0.829 0.618 US2 0.721 US3 0.789 User performance (UP) UP1 0.723 0.874 0.650 UP2 0.612 UP3 0.704 UP4 0.580 UP5 0.798 UP6 0.676 UP7 0.724 UP8 0.655 All loadings are statistically significant (p<0.01) Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 128 Table 4: Fornell-Larcker’s criterion ASSU EMP IQ PEOU PU REL RESP SQ UP US ASSU 0.776 EMP 0.512 0.790 IQ 0.495 0.519 0.828 PEOU 0.518 0.542 0.642 0.787 PU 0.462 0.312 0.504 0.542 0.843 REL 0.592 0.496 0.509 0.303 0.461 0.781 RESP 0.517 0.527 0.407 0.430 0.447 0.578 0.833 SQ 0.584 0.557 0.434 0.417 0.428 0.524 0.530 0.782 UP 0.557 0.435 0.420 0.532 0.320 0.506 0.429 0.334 0.806 US 0.582 0.550 0.462 0.320 0.416 0.538 0.544 0.320 0.548 0.786 The second approach is cross loading, “that is outer loading of each indicator on the associated construct should be higher than the cross loadings on the other construct” (Hair et al., 2017). As shown in Appendix A the values of the indicators have loaded highly on their respected construct. Thus, providing adequate evidence of fulfilling the convergent validity requirement. 5.2 Structural model assessment As per Hair et al. (2017) the structure model assessment includes: assessment of collinearity by evaluating the predictor constructs, assessing the significance and relevance of the path coefficients representing hypotheses among constructs, coefficient of determination (R2), effect size (f2), and assess the predictive relevance (Q2). For this study, SRQ construct is Higher Order Construct (HOC) with 4 sub-dimensions. HOC assessment is necessary to determine whether their first order (lower order) constructs load onto their respective second order (higher order) construct. To achieve HOC assessment, the researcher employed the two stage higher order construct modeling approach by (Becker et al., 2012; C. M. Ringle et al., 2012; Wilson, 2010). The first stage is repeated indicator approach. In this approach HOC’s measurement model is represented by assigning all the indicators of the first order construct to the HOC. The second stage is using latent variable scores as indicator representing the first order construct in the final structural model (Hair et al., 2017). The PLS-SEM produced latent variable scores for each construct and saved for further analysis. 5.2.1 Hypotheses tests Table 5 gives the hypothesized direct relationships results. Specifically, SQ (β = 0.474, t = 7.797, p< 0.01) has significance influence on PU. Similarly, SQ has significance relationship with PEOU (β = 0.381, t = 6.388, p< 0.01) and the results further confirmed the positive relationship between SQ and US (β = 0.396, t = 6.740, p< 0.01). Thus, providing support for H1, H2 and H3. Furthermore, IQ has significance influence on PU (β = 0.131, t = 3.124, p< 0.01), PEOU (β = 0.227, t = 4.441, p< 0.01) and US (β = 0.219, t = 4.302, p< 0.01). Therefore, H4, H5 and H6 are supported. Figure 2 shows structural model assessment. Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 129 Figure 2: PLS Algorithm of structural model Likewise, SRQ has positive and significance relationship with PU (β = 0.192, t = 3.017, p< 0.01), PEOU (β = 0.208, t = 3.546, p< 0.01) and US (β = 0.237, t = 3.933, p< 0.01). Hence, providing enough evidence to support hypotheses H7, H8, and H9. The results provide further indication that PU (β = 0.304, t = 9.760, p< 0.01) exhibits positive and significance influence on UP. In addition, PEOU (β = 0.227, t = 3.555, p< 0.01) has positive and significance effect on UP. Lastly, US (β = 0.439, t = 7.566, p< 0.01) shows significance effect on UP. Therefore, hypotheses H10, H11, and H12 are supported. Based on the Table 5, R2 value (0.826) for user performance imply that the combination of constructs: system quality, information quality, service quality, perceived usefulness, perceived ease of use and user satisfaction explain 82% of the variance in user performance. Similarly, system quality, Information quality and service quality jointly contribute 55% variance in perceived usefulness. The same set of predictors i.e. SQ, IQ, SRQ account for 56% and 60% variance in perceived ease of use and user satisfaction respectively. In conclusion, as per the recommendation of Chin (1998) the results shows that the predictive values for user performance can be considered as substantial, while perceived usefulness, perceived ease of use and user satisfaction qualify as moderate. Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 130 Table 5: Summary of results of hypotheses Hyp. Path () SE T value P Value Decision R2 f2 Q2 H1 SQ -> PU 0.476 0.061 7.797 0.000 Supported 0.553 0.133 0.360 H2 SQ -> PEOU 0.381 0.060 6.388 0.000 Supported 0.557 0.086 0.289 H3 SQ -> US 0.396 0.059 6.740 0.000 Supported 0.607 0.105 0.347 H4 IQ -> PU 0.131 0.042 3.124 0.001 Supported 0.017 H5 IQ -> PEOU 0.227 0.051 4.441 0.000 Supported 0.053 H6 IQ -> US 0.219 0.051 4.302 0.000 Supported 0.055 H7 SRQ -> PU 0.192 0.064 3.017 0.001 Supported 0.028 H8 SRQ -> PEOU 0.208 0.059 3.546 0.000 Supported 0.033 H9 SRQ -> US 0.237 0.060 3.933 0.000 Supported 0.048 H10 PU -> UP 0.304 0.031 9.760 0.000 Supported 0.826 0.208 0.345 H11 PEOU -> UP 0.227 0.064 3.555 0.000 Supported 0.032 H12 US -> UP 0.439 0.058 7.566 0.000 Supported 0.140 Note: p<0.01 The effect size (f2) for the study is also assessed. The effect size (f2) is “a measure used to assess the relative impact of a predictor construct on an endogenous construct” (Cohen, 1988). The results in Table 5 indicate that SQ has medium effects on PU (0.133), small effect on PEOU (0.086), and US (0.102). Information quality has small effects on PU (0.017) PEOU (0.053), US (0.055). Similarly, service quality exhibited small effect on PU (0.028), PEOU (0.033), and US (0.048). The result also indicated that the effect of PU on UP (0.208) is medium. Lastly, PEOU (0.032) and US (0.140) has small effect on UP. In terms of predictive relevance, the blindfolding procedure was employed to measure predictive relevance. Predictive relevance (Q2) represents “how well the data collected empirically can be reconstructed with the help of the model and the PLS parameters” (Fornell et al., 1994). As per Ramayah et al. (2018) and Hair et al. (2017) the recommended Q2 value should be larger than zero for dependent construct in structural model. The recommended Q2 values of 0.02 (small), 0.15 (medium) and 0.35 (large) indicate the level of predictive relevance for endogenous construct (Hair et al., 2017). Table 5 presents that the Q2 values for all endogenous constructs are above zero, clearly indicate that the model has predictive relevance. Thus, the result shows that one endogenous variable has a large effect and the remaining have medium predictive relevance. 6 Discussions The aim of the study is to evaluate the effect of quality factors on user performance. On integration of D&M IS success model and TAM, the study proposes and tests a framework. The result revealed that system quality positively effect perceived usefulness, perceived ease of use and user satisfaction. The effect of SQ on perceived usefulness (H1) is found significant and in line with studies of (Alfarraj et al., 2017; Ali et al., 2013; Cheng, 2019; Gorla et al., 2010; Lin, 2010; Tseng et al., 2018; Zhou, 2011, 2014). H2 is found supported, postulated as SQ positively effect PEOU. This relationship provides further support to the previous studies (Costa et al., 2016; Rana et al., 2015; Yang et al., 2017; Zhou, 2011). Another hypothesis as the effect on SQ on US (H3), was found supported and consistent with prior research in similar Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 131 context (Cheng, 2019; Masrek et al., 2016; Ojo, 2017; Tam et al., 2016). This imply that SQ is important antecedent to the perception of usefulness, ease of use and satisfaction. In this study context, system accuracy, system easiness, easy to learn are important considerations for users to perceive the system as useful, free of efforts and be satisfied. With regards to information quality relationships with PU, PEOU and US, three hypotheses H4, H5, H6 were tested. Consistent with previous research the influence of information quality on perceived usefulness (Alfarraj et al., 2017; Chen et al., 2015), perceived ease of use (Abugabah et al., 2015; Rana et al., 2015; Zhou, 2011) and user satisfaction (Chatterjee et al., 2018; Cheng, 2019; Hsu et al., 2015; Lin, 2010; Masrek et al., 2016; Sharma et al., 2019; Tam et al., 2016; Urbach et al., 2010) is confirmed. In this study context in higher education, users find the system as useful, ease of use when the obtained information from the system are clear, readable, well formatted and concise. Higher quality of information increases the satisfaction with the system by balancing the system output and user requirements. The relationship of service quality with perceived usefulness, perceived ease of use and user satisfaction were postulated as hypotheses H7, H8, H9 and found supported. The result supported the effect of SRQ on PU and confirmed the findings of (Ahn et al., 2007; Chen et al., 2015; Floropoulos et al., 2010; Landrum et al., 2010; Yang et al., 2017; Zhou, 2011). This imply that SRQ is key antecedent to PU. When users get high quality service from support staff, then the system is perceived as more useful. Similarly, in line with the studies of (Ahn et al., 2007; Lin, 2015) the result shows that SRQ has positive influence on PEOU. The system is perceived to be free of efforts when IT staff provide timely service, shows sincere efforts to solve their problems, and the services are reliable and error free. Moreover, the users will find the system as ease of use when IT staff boost their confidence, feels safe while dealing with IT staff, find the IT staff knowledgeable, and IT staff provide them individual attention. The result also reveals that SRQ has significant influence on user satisfaction. As expected the result supported previous studies (Floropoulos et al., 2010; Hsu et al., 2015; Landrum et al., 2010; Masrek et al., 2016; Mohammadi, 2015a; Noorman Masrek et al., 2010; Rana et al., 2015; Sharma et al., 2019; Tam et al., 2016). The findings imply that service quality strongly contributes to the satisfaction level of users upon providing services to them. In this study context, when the services provided by IT staff are reliable, timely and boost confidence to them, then users feel satisfaction with the system. This study also tested the effect of perceived usefulness, perceived ease of use and user satisfaction as antecedents to user performance and postulated as H10, H11, and H12. Consistent with the previous work (Abugabah et al., 2015; Ali et al., 2013; Chen et al., 2015), the effect of PU on user performance is confirmed. The result shows that PU is strong proxy for user performance, that is the perception of the system’s usefulness in terms of job performance. In the same vein the effect of perceived ease of use on UP is confirmed and Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 132 implies that the system perceived as free of efforts have impact on the performance on users. Lastly, the findings confirmed the significant influence of user satisfaction on user performance. Previously, positive effect of US on user performance was found in literature (A. H. Aldholay et al., 2018; Choi et al., 2013; Noorman Masrek et al., 2010). This implies that user satisfaction is a key factor of user performance, as satisfied users with the system leads to better performance. 7 Theoretical implications From the theoretical angle, this work provides several contributions in ERP system area. The first contribution is to bring together the quality factors of D&M IS success model and confirmed that each have effect on perceived usefulness, perceived ease of use and user satisfaction. Although, service quality has been part of updated D&M IS success model since 2003 but most of the research focuses on the relationships of SQ and IQ with different constructs to measure system success. In current study service quality is used to captures user’s assessment of services delivered by IT staff as they collect holistic view of service quality while interacting with them. Second, the study incorporates perceived usefulness, perceived ease of use and user satisfaction. In ERP systems evaluation studies these effects are tested singularly or in combination involving other antecedents. Thus, according to researcher knowledge, this is the first empirical work to measure the impact of the ERP systems on through PU, PEOU and US in a framework involving SQ, IQ, and SRQ as their antecedents. This confirms the importance of TAM as PU and PEOU facilitates user performance in the presence of quality factors in this study context. Hence, incorporating ERP quality factors of D&M IS success model with PU and PEOU of TAM can capture complete picture to evaluate ERP system. Contextually, the work generally contributes to the body of knowledge on user performance and particularly to higher education from the perspective of users. Mostly, in higher education the focus remained on technical aspects of the systems. Thus, the study contributes to literature on higher education, especially higher education of Pakistan. 8 Practical implications In terms of practical contribution, one of the implications is the establishment that perceived usefulness and perceived ease of use were shown vital impact on user performance. This shows that users put emphasis on usefulness and easy to use. This provides practitioners the opportunity to help users to realize the benefits of the system with regards to usefulness and ease of use. Therefore, the practitioners need to put efforts to provide a system which user perceive as useful and free of efforts, that is to produce a system that make their job easy, to enhance their productivity, easy to use, easy to learn, understandable. Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 133 In case of user satisfaction, it can be concluded that US plays the role between the relationship of system quality and information quality and service quality with user performance. Hence, the practitioners need to ensure that users are satisfied with the functional features, output of the system and services provided to them. Another major conclusion is that among all quality predictors the results suggest that system quality is having great impact. The possible conclusion is that users place more emphasis on system quality which represent functional features of the system. The practitioners need to put efforts on the functional features of the system. 9 Limitations and future recommendations The first the limitation to be considered is that the study proposes and tests a new framework in ERP environment. The data for this scholarly work was collected and analyzed from a subsect of universities of Pakistan. Regardless of the significant relationships among constructs, these results may not be generalized to other sectors. The constructs used in this study and their relationships can be used for further investigation in different industries. Second, the framework of the current study is based on the constructs from D & M IS success model and TAM. Therefore, other factors related to users such as user resistance, user characteristics influencing user performance may be explored by extended this framework. Third, the target population were the users regardless of their designations. Hence, researchers may target users in different layers such as academic and non-academic staff. 10 Conclusions Upon the integration of the D&M IS success model and TAM, the study proposes and tested a framework to test system’s effect on users. The framework clarifies how ERP quality factors (system quality, information quality, service quality) influence perceived usefulness, perceived ease of use and user satisfaction from user’s perspective. The study goes further to investigate the effect of perceived usefulness, perceive ease of use and user satisfaction on user performance. The results showed that significance influence exists from exogenous constructs on user performance in ERP system environment. The findings also established the importance of perceived usefulness and perceived ease of use representing TAM’s portion in ERP environment in higher education’s institutions of Pakistan. References Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 134 Abugabah, A., Sanzogni, L., & Alfarraj, O. (2015). Evaluating the impact of ERP systems in higher education. The International Journal of Information and Learning Technology, 32(1), 45-64. Ahmad, N., Haleem, A., & Ali Syed, A. (2014). Study of reasons for enterprise systems adoption among Indian organizations. Journal of Enterprise Information Management, 27(6), 696-718. Ahmer, Z., Demir, E., Tofallis, C., & Asad, H. (2018). Usage of enterprise resource planning systems in higher education institutions in Pakistan. doi:https://uhra.herts.ac.uk/handle/2299/19625 Ahn, T., Ryu, S., & Han, I. (2007). The impact of Web quality and playfulness on user acceptance of online retailing. Information & Management, 44(3), 263-275. Albarghouthi, M., Qi, B., Wang, T. C., & Abbad, M. (2020). ERP Adoption and Acceptance in Saudi Arabia Higher Education: A Conceptual Model Development. International Journal of Emerging Technologies in Learning(15). Aldholay, A., Isaac, O., Abdullah, Z., Abdulsalam, R., & Al-Shibami, A. H. (2018). An extension of Delone and McLean IS success model with self-efficacy: Online learning usage in Yemen. The International Journal of Information and Learning Technology, 35(4), 285-304. Aldholay, A. H., Isaac, O., Abdullah, Z., & Ramayah, T. (2018). The role of transformational leadership as a mediating variable in DeLone and McLean information system success model: The context of online learning usage in Yemen. Telematics and Informatics, 35(5), 1421-1437. Alfarraj, O., & Abugabah, A. (2017). Extending information system models to the health care context: an empirical study and experience from developing countries. International Arab Journal of Information Technology, 14(2), 159-167. Ali, B. M., & Younes, B. (2013). The impact of information systems on user performance: an exploratory study. Journal of Knowledge Management, Economics and Information Technology, 3(2), 128-154. Almahamid, S., & Awsi, O. (2015). Perceived organizational ERP benefits for SMEs: Middle Eastern perspective. Interdisciplinary Journal of Information, Knowledge, and Management, 10. Alsaleh, I., & Bageel, M. (2016). Measuring User Satisfaction with Service Quality of IT Department Support as Perceived by the Users: Case Study of Service Industry Sector in Jeddah, Saudi Arabia. International Journal of Liberal Arts and Social Science, 4(1), 65-82. AlShamlan, H. M., & AlMudimigh, A. S. (2011). The Chang management strategies and processes for successful ERP implementation: a case study of MADAR. International Journal of Computer Science, 8(2), 399-407. Alshibly, H. H. (2014). Evaluating E-HRM success: A Validation of the Information Systems Success Model. International Journal of Human Resource Studies, 4(3), 107. Arif, M., Kulonda, D. J., Proctor, M., & Williams, K. (2004). Before you invest: An illustrated framework to compare conceptual designs for an enterprise information system. Information Knowledge Systems Management, 4(2), 119-135. Ayman, B., & Kamaljeet, S. (2015). Factors Influencing The Acceptance Of Enterprise Resource Planning System (ERP) And Financial Performance Of Saudi Arabia Listed Companies: Multivariate Data Analysis Using Structural Equation Modeling (SEM). International Journal of Business and Management Review, 3(4), 93-118. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory: Englewood Cliffs, NJ, US: Prentice-Hall, Inc. Banker, R. D., Bardhan, I. R., Chang, H., & Lin, S. (2006). Plant information systems, manufacturing capabilities, and plant performance. MIS quarterly, 315-337. Becker, J.-M., Klein, K., & Wetzels, M. (2012). Hierarchical latent variable models in PLS-SEM: guidelines for using reflective-formative type models. Long range planning, 45(5-6), 359-394. Beheshti, H. M., & Beheshti, C. M. (2010). Improving productivity and firm performance with enterprise resource planning. Enterprise Information Systems, 4(4), 445-472. https://uhra.herts.ac.uk/handle/2299/19625 Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 135 Chatterjee, S., Kar, A. K., & Gupta, M. (2018). Success of IoT in Smart Cities of India: An empirical analysis. Government Information Quarterly. Chen, J. V., Jubilado, R. J. M., Capistrano, E. P. S., & Yen, D. C. (2015). Factors affecting online tax filing–An application of the IS Success Model and trust theory. Computers in Human Behavior, 43, 251-262. Cheng, Y.-M. (2019). A hybrid model for exploring the antecedents of cloud ERP continuance: Roles of quality determinants and task-technology fit. International Journal of Web Information Systems, 15(2), 215-235. Chin, W. W. (1998). Issues and Opinion on structural Equation Modelling. Management Information Systems quarterly, 22(1), 1-8. Choi, W., Rho, M. J., Park, J., Kim, K. J., Kwon, Y. D., & Choi, I. Y. (2013). Information system success model for customer relationship management system in health promotion centers. Healthcare informatics research, 19(2), 110-120. Chou, S.-W., & Chang, Y.-C. (2008). The implementation factors that influence the ERP (enterprise resource planning) benefits. Decision Support Systems, 46(1), 149-157. Cohen, J. (1988). Statistical Power Analysis for the Behavioral SciencesNew JerseyLawrence Erlbaum Associates. Inc. Publishers. Costa, C. J., Ferreira, E., Bento, F., & Aparicio, M. (2016). Enterprise resource planning adoption and satisfaction determinants. Computers in Human Behavior, 63, 659-671. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 319-340. DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information systems research, 3(1), 60-95. Delone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: a ten-year update. Journal of management information systems, 19(4), 9-30. Floropoulos, J., Spathis, C., Halvatzis, D., & Tsipouridou, M. (2010). Measuring the success of the Greek taxation information system. International Journal of Information Management, 30(1), 47-56. Fornell, C., & Cha, J. (1994). Partial Least Squares. Advanced methods of marketing research, 407(3), 52-78. Fornell, C., & Larcker, D. F. (1981). Structural equation models with unobservable variables and measurement error: Algebra and statistics. Journal of Marketing Research, 382-388. Gable, G. G., Sedera, D., & Chan, T. (2008). Re-conceptualizing Information System Success: The IS- Impact Measurement Model. Journal of the Association for Information Systems, 9(7), 377- 408. Galy, E., & Sauceda, M. J. (2014). Post-implementation practices of ERP systems and their relationship to financial performance. Information & Management, 51(3), 310-319. Gefen, D., Straub, D., & Boudreau, M.-C. (2000). Structural equation modeling and regression: Guidelines for research practice. Communications of the Association for Information Systems, 4(1), 7. Gill, A. A., Amin, S., & Saleem, A. (2020). Investigation of Critical Factors for Successful ERP Implementation: An Exploratory Study. Journal of Business and Social Review in Emerging Economies, 6(2), 565-575. Gorla, N., & Somers, T. M. (2014). The impact of IT outsourcing on information systems success. Information & Management, 51(3), 320-335. Gorla, N., Somers, T. M., & Wong, B. (2010). Organizational impact of system quality, information quality, and service quality. The Journal of Strategic Information Systems, 19(3), 207-228. Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 136 Hair, J. F., Anderson, R. E., Babin, B. J., & Black, W. C. (2010). Multivariate data analysis: A global perspective (Vol. 7): Pearson Upper Saddle River, NJ. Hair, J. F., Hult, G. T. M., Ringle, C., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM) (2nd Edition ed.): Sage Publications. Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the academy of marketing science, 43(1), 115-135. Hsieh, P., Huang, C., & Yen, D. C. (2013). Assessing web services of emerging economies in an Eastern country Taiwan's e-government. Government Information Quarterly, 30(3), 267-276. Hsu, P.-F., Yen, H. R., & Chung, J.-C. (2015). Assessing ERP post-implementation success at the individual level: Revisiting the role of service quality. Information & Management, 52(8), 925- 942. Ifinedo, P., Rapp, B., Ifinedo, A., & Sundberg, K. (2010). Relationships among ERP post- implementation success constructs: An analysis at the organizational level. Computers in Human Behavior, 26(5), 1136-1148. Isaac, O., Abdullah, Z., Ramayah, T., & Mutahar, A. M. (2017). Internet usage, user satisfaction, task- technology fit, and performance impact among public sector employees in Yemen. The International Journal of Information and Learning Technology, 34(3), 210-241. Kline, R. B. (2010). Principles and Practice of Structural Equation Modeling: Guilford press. Kumar, M., Talib, S. A., & Ramayah, T. (2013). Business Research Methods. Oxford: Oxford University Press. Landrum, H., Prybutok, V. R., & Zhang, X. (2010). The moderating effect of occupation on the perception of information services quality and success. Computers & Industrial Engineering, 58(1), 133-142. Lin, H. F. (2010). An investigation into the effects of IS quality and top management support on ERP system usage. Total Quality Management, 21(3), 335-349. Lin, H. F. (2015). The impact of company-dependent and company-independent information sources on organizational attractiveness perceptions. Journal of Management Development, 34(8), 941- 959. Liu, L., Feng, Y., Hu, Q., & Huang, X. (2011). From transactional user to VIP: how organizational and cognitive factors affect ERP assimilation at individual level. European Journal of Information Systems, 20(2), 186-200. Masrek, M. N., & Gaskin, J. E. (2016). Assessing users satisfaction with web digital library: the case of Universiti Teknologi MARA. The International Journal of Information and Learning Technology, 33(1), 36-56. Mohammadi, H. (2015a). Factors affecting the e-learning outcomes: An integration of TAM and IS success model. Telematics and Informatics, 32(4), 701-719. Mohammadi, H. (2015b). Investigating users’ perspectives on e-learning: An integration of TAM and IS success model. Computers in Human Behavior, 45, 359-374. Montesdioca, G. P. Z., & Maçada, A. C. G. (2015). Measuring user satisfaction with information security practices. Computers & security, 48, 267-280. Nelson, R. R., Todd, P. A., & Wixom, B. H. (2005). Antecedents of information and system quality: an empirical examination within the context of data warehousing. Journal of management information systems, 21(4), 199-235. Nizamani, S., Khoumbati, K., Ismaili, I. A., & Nizamani, S. (2014). A Conceptual Framework for ERP Evaluation in Universities of Pakistan. Sindh University Research Journal, 45(3), 467-475. Noorman Masrek, M., Jamaludin, A., & Awang Mukhtar, S. (2010). Evaluating academic library portal effectiveness: A Malaysian case study. Library Review, 59(3), 198-212. Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 137 Ojo, A. I. (2017). Validation of the DeLone and McLean Information Systems Success Model. Healthcare informatics research, 23(1), 60-66. Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1988). Servqual: A multiple-item scale for measuring consumer perc. Journal of retailing, 64(1), 12. Petter, S., DeLone, W., & McLean, E. (2008). Measuring information systems success: models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17(3), 236-263. Petter, S., & McLean, E. R. (2009). A meta-analytic assessment of the DeLone and McLean IS success model: An examination of IS success at the individual level. Information & Management, 46(3), 159-166. Pitt, L. F., Watson, R. T., & Kavan, C. B. (1995). Service quality: a measure of information systems effectiveness. MIS quarterly, 173-187. Rabaa'i, A. A., Bandara, W., & Gable, G. (2009). ERP systems in the higher education sector: a descriptive study. Paper presented at the 20th Australasian Conference on Information Systems. Rai, A., Lang, S. S., & Welker, R. B. (2002). Assessing the validity of IS success models: An empirical test and theoretical analysis. Information systems research, 13(1), 50-69. Rajan, C. A., & Baral, R. (2015). Adoption of ERP system: An empirical study of factors influencing the usage of ERP and its impact on end user. IIMB Management Review, 27(2), 105-117. Ram, J., Wu, M.-L., & Tagg, R. (2014). Competitive advantage from ERP projects: Examining the role of key implementation drivers. International Journal of Project Management, 32(4), 663-675. Ramayah, T., Cheah, J., Chuah, F., Ting, H., & Mumtaz, A. M. (2018). Partial least squares structural equation modeling (PLS-SEM) using SmartPLS 3.0: An updated and practical guide to statistical analysis. (2nd Edition ed.). Pearson, Malaysia. Rana, N. P., Dwivedi, Y. K., Williams, M. D., & Weerakkody, V. (2015). Investigating success of an e-government initiative: validation of an integrated IS success model. Information systems frontiers, 17(1), 127-142. Ranganathan, C., & Brown, C. V. (2006). ERP investments and the market value of firms: Toward an understanding of influential ERP project variables. Information systems research, 17(2), 145- 161. Ringle, C., Wende, S., & Becker, J.-M. (2015). B. SmartPLS GmbH. Ringle, C. M., Sarstedt, M., Mitchell, R., & Gudergan, S. P. (2020). Partial least squares structural equation modeling in HRM research. The International Journal of Human Resource Management, 31(12), 1617-1643. Ringle, C. M., Sarstedt, M., & Straub, D. (2012). A critical look at the use of PLS-SEM in MIS Quarterly. Rogers, E. M. (1995). Diffusion of innovations. New York: Simon and Schuster. Shahibi, M. S., Saidin, A., & Izhar, T. A. T. (2016). Evaluating User Satisfaction on Human Resource Management Information System (HRMIS): A Case of Kuala Lumpur City Hall, Malaysia. International Journal of Academic Research in Business and Social Sciences, 6(10), 95-116. Sharma, S. K., & Sharma, M. (2019). Examining the role of trust and quality dimensions in the actual usage of mobile banking services: An empirical investigation. International Journal of Information Management, 44, 65-75. Shen, Y.-C., Chen, P.-S., & Wang, C.-H. (2016). A study of enterprise resource planning (ERP) system performance measurement using the quantitative balanced scorecard approach. Computers in Industry, 75, 127-139. Solutions, P. C. (2015). ERP Report: a Panorama Consulting Solutions research report. Retrieved from Sternad, S., & Bobek, S. (2013). Impacts of TAM-based external factors on ERP acceptance. Procedia Technology, 9, 33-42. Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 138 Sun, H., Ni, W., & Lam, R. (2015). A step-by-step performance assessment and improvement method for ERP implementation: Action case studies in Chinese companies. Computers in Industry, 68, 40-52. Swartz, D., & Orgill, K. (2001). Higher education ERP: Lessons learned. EDUCAUSE Quarterly, 24(2), 20-27. Tam, C., & Oliveira, T. (2016). Understanding the impact of m-banking on individual performance: DeLone & McLean and TTF perspective. Computers in Human Behavior, 61, 233-244. Tsai, W.-H., Lee, P.-L., Shen, Y.-S., & Lin, H.-L. (2012). A comprehensive study of the relationship between enterprise resource planning selection criteria and enterprise resource planning system success. Information & Management, 49(1), 36-46. Tseng, T. H., & Lee, C. T. (2018). Facilitation of consumer loyalty toward branded applications: The dual-route perspective. Telematics and Informatics, 35(5), 1297-1309. Ullah, A., Baharun, R. B., Nor, K., Siddique, M., & Bhatti, M. N. (2018). Enterprise Resource Planning (ERP) Systems and ERP Quality Factors: A Literature Review. Journal of Managerial Sciences, 11(3), 297-322. Ullah, A., Baharun, R. B., Nor, K., Siddique, M., & Sami, A. (2018). Enterprise Resource Planning (ERP) Systems and User Performance (UP). International Journal of Applied Decision Sciences, 377-390. Umar, M., Khan, N., Agha, M., & Abbas, M. (2016). Exploring the Factors Affecting Enterprise Resource Planning (ERP) Implementation Quality. Journal of Quality and Technology Management, 7(1), 137-155. Urbach, N., Smolnik, S., & Riempp, G. (2010). An empirical investigation of employee portal success. The Journal of Strategic Information Systems, 19(3), 184-206. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS quarterly, 425-478. Wang, E. S.-T. (2016). The moderating role of consumer characteristics in the relationship between website quality and perceived usefulness. International Journal of Retail & Distribution Management, 44(6), 627-639. Wang, Y.-S., & Liao, Y.-W. (2008). Assessing eGovernment systems success: A validation of the DeLone and McLean model of information systems success. Government Information Quarterly, 25(4), 717-733. Wang, Y. S. (2008). Assessing e‐commerce systems success: a respecification and validation of the DeLone and McLean model of IS success. Information systems journal, 18(5), 529-557. Wilson, B. (2010). Using PLS to investigate interaction effects between higher order branding constructs Handbook of partial least squares (pp. 621-652): Springer. Xu, X., Zhang, W., & Barkhi, R. (2010). IT infrastructure capabilities and IT project success: a development team perspective. Information Technology and Management, 11(3), 123-142. Yang, M., Shao, Z., Liu, Q., & Liu, C. (2017). Understanding the quality factors that influence the continuance intention of students toward participation in MOOCs. Educational Technology Research and Development, 65(5), 1195-1214. Zhou, T. (2011). Examining the critical success factors of mobile website adoption. Online Information Review, 35(4), 636-652. Zhou, T. (2014). An empirical examination of initial trust in mobile payment. Wireless personal communications, 77(2), 1519-1531. Zhu, Y., Li, Y., Wang, W., & Chen, J. (2010). What leads to post-implementation success of ERP? An empirical study of the Chinese retail industry. International Journal of Information Management, 30(3), 265-276. Appendix B Questionnaire items Journal of Applied Economics and Business Studies, Volume. 4, Issue 2 (2020) 119-140 https://doi.org/10.34260/jaebs.426 139 System quality (Hsu et al., 2015); (Gable et al., 2008); (Nelson et al., 2005) SQ1 The CMS system is easy to use SQ2 The CMS system is easy to learn SQ3 The CMS system always processes data accurately SQ4 The CMS system requires only a minimum number of fields and screens to complete a task SQ5 The CMS system meets my requirements. SQ6 The CMS system includes necessary features and functions for my job. SQ7 The CMS system user interface can be easily adapted to my personal approach. SQ8 All the data that I use within the CMS system are fully integrated and consistent SQ9 The CMS system can be easily modified or improved according to my needs. Information Quality (Hsu et al., 2015); (Gable et al., 2008); (Nelson et al., 2005) IQ1 The CMS system provides output that is exactly what I want IQ2 Information needed from the CMS system is always available IQ3 Information from the CMS system is in a form that is readily usable IQ4 Information from the CMS system is easy to understand IQ5 Information from the CMS system appears readable, clear, and well formatted IQ6 Information from CMS system is concise Service quality (Hsu et al., 2015); (Pitt et al., 1995); (Parasuraman et al., 1988) Reliability SRQ1 The IT department provides its services at the time it promises to do SRQ2 When users have a problem, the IT staff shows a sincere interest in solving it SRQ3 The IT department is reliable SRQ4 The IT department insists on error-free records Responsiveness SRQ5 The IT staff informs users exactly when services will be performed SRQ6 The IT staff gives prompt service to users. SRQ7 The staff of the IT department is never too busy to respond to user’s requests. Assurance SRQ8 The behaviour of the staff in IT department boosts the confidence of users. SRQ9 I feel safe in my dealings with the IT staff SRQ10 IT staff is consistently courteous with users. SRQ11 The IT staff have the knowledge to do their job well Empathy SRQ12 The IT department has operating hours convenient to all users. SRQ13 The IT department gives users individual attention SRQ14 The IT department has the users’ best interests at heart SRQ15 The staff of the IT department understand the specific needs of the users Perceived usefulness (Abugabah et al., 2015) PU1 Using the CMS system improves my performance in my job PU2 Using the CMS system in my job increases my productivity PU3 Using the CMS system enhances my effectiveness in my job PU4 I find the system to be useful in my job Perceived ease of use (Abugabah et al., 2015) PEOU1 I find the CMS system easy to use PEOU2 I find it easy to get the CMS system to do what I want it to do PEOU3 My interaction with the CMS system is clear and understandable. PEOU4 Interacting with the system does not require a lot of my mental efforts User satisfaction (Lin, 2010) US1 The information I get from CMS system is very satisfying US2 My interaction with CMS system is very satisfying US3 Overall, I am very satisfied with CMS system User performance (Abugabah et al., 2015) UP1 The quality of the CMS system enables me to accomplish my work UP2 Our CMS system has a positive impact on my productivity UP3 Using CMS system in my job enables me to accomplish multiple tasks more quickly UP4 Overall, our CMS system improves my efficiency in my job UP5 Our CMS system helps me solve my job problems UP6 Our CMS system reduces performance errors in my job UP7 Our CMS system enhances my effectiveness in my job Abrar Ullah, Rohaizat Bin Baharun, & Muhammad Yasir, Khalil MD Nor 140 UP8 Our CMS system helps me create new ideas in my job UP9 Our CMS system enhances my creativity UP10 Overall, our ERP system helps me achieve my job goals Appendix C Cross loadings SQ IQ REL RESP ASSU EMP PU PEOU US UP SQ2 0.686 0.489 0.452 0.505 0.486 0.442 0.480 0.476 0.490 0.401 SQ3 0.777 0.471 0.482 0.548 0.527 0.468 0.406 0.531 0.541 0.451 SQ4 0.791 0.527 0.538 0.605 0.584 0.492 0.402 0.530 0.547 0.468 SQ5 0.781 0.558 0.464 0.527 0.529 0.511 0.582 0.533 0.492 0.471 SQ6 0.650 0.532 0.424 0.511 0.472 0.489 0.458 0.498 0.534 0.531 SQ8 0.727 0.586 0.425 0.516 0.447 0.492 0.465 0.492 0.500 0.574 SQ9 0.785 0.473 0.463 0.588 0.510 0.533 0.578 0.546 0.570 0.589 IQ1 0.412 0.776 0.454 0.578 0.487 0.516 0.581 0.590 0.409 0.445 IQ2 0.421 0.819 0.455 0.527 0.443 0.371 0.459 0.526 0.544 0.595 IQ3 0.561 0.849 0.371 0.477 0.311 0.382 0.491 0.544 0.540 0.565 IQ4 0.401 0.874 0.451 0.481 0.419 0.436 0.470 0.504 0.544 0.590 IQ5 0.582 0.851 0.417 0.471 0.407 0.413 0.456 0.509 0.544 0.509 IQ6 0.553 0.794 0.370 0.461 0.372 0.442 0.528 0.493 0.489 0.570 SRQ1 0.422 0.296 0.662 0.375 0.468 0.392 0.344 0.403 0.429 0.456 SRQ2 0.417 0.365 0.750 0.351 0.363 0.305 0.286 0.299 0.329 0.365 SRQ3 0.553 0.472 0.837 0.553 0.493 0.413 0.399 0.421 0.446 0.524 SRQ5 0.522 0.418 0.534 0.813 0.485 0.479 0.426 0.478 0.490 0.557 SRQ6 0.471 0.569 0.501 0.864 0.539 0.518 0.315 0.496 0.408 0.478 SRQ7 0.427 0.526 0.411 0.821 0.517 0.571 0.571 0.499 0.512 0.496 SRQ9 0.372 0.204 0.512 0.362 0.693 0.352 0.141 0.228 0.314 0.346 SRQ10 0.408 0.407 0.449 0.578 0.802 0.520 0.492 0.492 0.519 0.581 SRQ11 0.569 0.502 0.417 0.461 0.802 0.521 0.390 0.441 0.486 0.552 SRQ12 0.369 0.254 0.356 0.375 0.500 0.740 0.287 0.245 0.269 0.352 SRQ13 0.492 0.498 0.371 0.496 0.424 0.779 0.514 0.472 0.486 0.519 SRQ14 0.369 0.463 0.443 0.595 0.527 0.847 0.317 0.542 0.528 0.321 PU1 0.590 0.476 0.350 0.498 0.358 0.504 0.813 0.541 0.530 0.410 PU2 0.580 0.472 0.384 0.537 0.368 0.491 0.841 0.553 0.532 0.511 PU3 0.561 0.525 0.386 0.575 0.398 0.553 0.880 0.437 0.593 0.474 PU4 0.520 0.556 0.429 0.564 0.426 0.512 0.836 0.560 0.389 0.498 PEOU1 0.493 0.429 0.350 0.454 0.380 0.396 0.421 0.686 0.463 0.455 PEOU2 0.524 0.468 0.396 0.395 0.442 0.300 0.329 0.621 0.421 0.494 PEOU3 0.420 0.556 0.429 0.564 0.426 0.512 0.436 0.860 0.489 0.598 PEOU4 0.516 0.471 0.338 0.468 0.319 0.401 0.685 0.820 0.543 0.548 US1 0.613 0.530 0.442 0.539 0.508 0.456 0.496 0.397 0.843 0.524 US2 0.524 0.468 0.396 0.395 0.442 0.300 0.329 0.321 0.721 0.494 US3 0.520 0.556 0.429 0.564 0.426 0.512 0.533 0.586 0.789 0.440 UP1 0.562 0.513 0.574 0.500 0.546 0.532 0.559 0.543 0.539 0.723 UP2 0.536 0.510 0.332 0.443 0.400 0.375 0.493 0.404 0.377 0.612 UP3 0.594 0.524 0.419 0.475 0.456 0.380 0.540 0.482 0.481 0.704 UP4 0.448 0.436 0.395 0.397 0.459 0.383 0.377 0.436 0.451 0.580 UP5 0.520 0.556 0.429 0.564 0.426 0.512 0.436 0.350 0.489 0.798 UP6 0.567 0.478 0.387 0.547 0.457 0.468 0.587 0.501 0.456 0.676 UP7 0.313 0.530 0.442 0.539 0.508 0.456 0.496 0.397 0.443 0.724 UP8 0.493 0.429 0.350 0.454 0.380 0.396 0.421 0.486 0.463 0.655