Microsoft Word - 31-3159_s_ETASR_V9_N6_pp5037-5040 Engineering, Technology & Applied Science Research Vol. 9, No. 6, 2019, 5037-5040 5037 www.etasr.com Hafeez et al.: A Survey on the Adaption of CMS in Pakistani Universities A Survey on the Adaption of CMS in Pakistani Universities Abdul Hafeez Department of Computer Science SMI University Karachi, Pakistan Samreen Javed Department of Computer Science SMI University Karachi, Pakistan Anus Bin Murtaza Department of Computer Science Muhammad Ali Jinnah University Karachi, Pakistan Abdul Aziz Department of Computer Science National University of Computer and Emerging Sciences, Karachi, Pakistan Syed Muhammad Hassan Department of Computer Science SMI University Karachi, Pakistan Imtiaz Hussain Department of Computer Science SMI University Karachi, Pakistan Abstract—The use of Campus Management Systems (CMSs) has increased dramatically in Pakistani universities, a huge amount of money has been invested in its development and deployment. The CMS provides an integrated platform for managing academic activities, controlling process flows and provide online access to related information. It improves the efficiency and effectiveness of the universities and eventually improves the quality of teaching. However, it is very important to consider the attitudes and perceptions of faculty members and students towards the adoption of CMS because they may affect the acceptance of this technology. The aim of this study is to investigate and highlight the user satisfaction level about the CMS quality according to ISO/IEC 9126 standard. This work uses cross-sectional design as a primary research method, and data from 105 students and faculty members were collected from a Pakistani public sector university using a questionnaire survey. The response of the respondents illustrates that the system functionality is good, reliable, usable, and efficient. However, some improvements are necessary in some areas such as understandability, learnability, operability, and attractiveness. Keywords-software quality; software functionality, CMS I. INTRODUCTION Information Technology applications play a significant role in any organization or company [1]. They are controlling robots, helping in healthcare and commerce [2-4], they have also reached higher education in recent years under the name of CMS [5] which basically are Enterprise Recourse Planning (ERP) programs for institutions [6, 7]. The CMS improves the efficiency and effectiveness of the overall organization and ultimately improves the quality of teaching and learning. It shows the holistic picture of university processes [7]. In developing countries, public and private sector universities are facing competition in the higher education market, and want to use information and communication technology (ICT) as an edge on other institutions [8]. In the ICT landscape system, student-oriented processes are important because they reduce the operational cost, which is done when students independently manage their academic activities such as enrollment or acquiring information such as attendance and grades. Through the integration of CMS and Web, these services are available any time. The importance of CMS has also been recognized by the accreditation of institutions such as the Higher Education Commission (HEC) of Pakistan which initiated the HEC-funded deployment of CMS on different public sector universities [9]. Various universities in Pakistan are implementing CMS to handle their academic processes Assessment and evaluation are conducted to improve the efficiency and effectiveness of any system. There are different kinds of evaluations such as (1) formative, which is performed during the development and (2) summative, which is performed after the completion of the system. It focuses on system’s effectiveness and if it meets the original requirements. This work evaluates the CMS in four characteristics: functionality, reliability, usability, and efficiency. These characteristics are basically a part of ISO/IEC 9126-1 standard [10, 11] which focuses on 6 characteristics of software quality. However, the remaining two, portability and maintainability are not directly related to the end-users (students and faculty members). Only CMS features related to students and faculty such as student records, gradebook, student self-service, and faculty self- service were considered in the current survey. A. Functionality The functionality of the system concentrates the complete function list that should be provided by it. Table I shows the subcharacteristics of functionality and the respected questions asked to the end-users. B. Reliability Reliability concerns the probability of error-free operation of the software and its ability to maintain its service delivery under specified circumstance. Table II shows the sub- characteristics of reliability and the respected questions asked to the end-users. Corresponding author: Abdul Hafeez (ahkhan@smiu.edu.pk) Engineering, Technology & Applied Science Research Vol. 9, No. 6, 2019, 5037-5040 5038 www.etasr.com Hafeez et al.: A Survey on the Adaption of CMS in Pakistani Universities TABLE I. FUNCTIONALITY SUB-CHARACTERISTICS Characteristic Sub-characteristics Question Functionality Suitability Can the CMS do the required tasks? Interoperability Does the CMS interact with another system? Security Does the CMS prevent unauthorized access? Accuracy Does the CMS give results as expected? TABLE II. RELIABILITY SUB-CHARACTERISTICS Characteristic Sub-characteristics Questions Reliability Maturity Has a maximum number of errors in the CMS been removed over time? Fault tolerance Is CMS able to handle errors? Recoverability After failure, can the CMS continue working and recover lost data? C. Usability Usability is concerned about the software system’s ease of use and whether it is learnable. In usability, we focus on the user interface and how much the software system interface is easy to operate, easy to learn, and attractive to the end-users. Table III shows the sub characteristics of usability and the respected questions asked to the end-users. TABLE III. USABILITY SUB-CHARACTERISTICS Characteristic Sub-characteristics Questions Usability Understandability Does the user apprehend how to easily use the system? Learnability Can the end-user learn the system easily? Operability Can the user use the system without much effort? Attractiveness Does the system has a good look and feel? D. Efficiency The efficiency concerns the resources’ utilization when performing the required operations such as memory, amount of disk space, processor, network, etc. Table IV shows the sub- characteristics of efficiency and the respected questions asked to the end-users. TABLE IV. EFFICIENCY SUB-CHARACTERISTICS Characteristics Sub-characteristics Questions Efficiency Time behavior How rapidly does the system respond and answer to queries? Resource utilization Does the system efficiently utilize system resources? II. METHODOLOGY A. Research Method There are various methods for evaluating software quality but selecting the correct one for evaluation depends on complexity and functionality. This study used the qualitative method for data collection and in order to understand the participants’ attitudes and experience. It is used in order to evaluate ideas, experience, and beliefs of the participants regarding the quality of the CMS. B. Participants In the survey, data were collected from N=105 students and faculty members. In the sample, 74% participants were male and 24% were female, while 20% were faculty members and 80% were students, out of which 59% were in their 4th year, 34% in their 3rd year and the rest were in their 2nd year, while the 1st year students were not selected due to their little experience on CMS. C. Data Collection Tools In this study, the evaluation form included two sections. The first section included demographic information, such as gender, age, and department, and the 2nd section consisted of the survey questions. The results showed that this method of assessment is effective in pointing various software quality issues in the CMS. D. Data Analysis Data were analyzed with SPSS to find how much the CMS realized the four considered characteristics of the ISO/IEC 9126-1. The feedbacks of all sub-characteristic were combined, and the basic mean was obtained. E. Validity and Reliability In this study, for ensuring reliability, the data were collected through according to ethical research. In order to reach the utilization in the study, the faculty and students of Pakistani university who experienced CMS were selected. Fig. 1. Percentage distribution of CMS product quality characteristics feedback TABLE V. DESCRIPTIVE STATISTICS Quality characteristics N Min Max Mean SD Functionality 105 27 160 84 50.3 Reliability 35 98 63 27.1 Usability 50 126 84 32.5 Efficiency 17 62 42 17.4 0 5 10 15 20 25 30 35 40 Strongly Agree Agree May Be Disagree Strongly Disagree CMS Product Quality Characteristics Functionality Reliability Usability Efficency Engineering, Technology & Applied Science Research Vol. 9, No. 6, 2019, 5037-5040 5039 www.etasr.com Hafeez et al.: A Survey on the Adaption of CMS in Pakistani Universities F. Results and Discussion In this section, the data collected from the questionnaire survey are presented and analyzed statistically. In this work, a questionnaire was spread among faculty members and students who were familiar with the CMS. In addition, interviews with some faculty members and students were conducted in order to validate the gaps in the questionnaire. Figure 1 shows the detailed feedback regarding the quality characteristics of the CMS. Four items were combined to determine this factor, and the results are shown in Table V. The minimum and maximum scores were 27 and 160, respectively. As it can be seen from Table V, the overall mean is 84. So, it can be concluded that the faculty and students were satisfied with the functionality of the CMS. Table VI and Figure 2 show the detail feedbacks on how the functionality of the CMS works. As it can be seen in Table VI, around 51.5% of the participants’ responses fall in the range of agree and strongly agree which shows satisfaction on the functionality of the studied system. It is worth mentioning that 23.8% of the participants were neutral on their response on the functionality of the system and 25.1% of the participants were unhappy with the functionality of CMS and gave disagree and strongly disagree feedback. Fig. 2. Percentage distribution of CMS functionality characteristics feedback Fig. 3. Percentage distribution of CMS reliability characteristics feedback The reliability of the system was examined and the results are shown in Table V. The minimum and maximum scores are 35 and 98 respectively and the mean is 63. It can be concluded that the faculty and students were not very much satisfied with the systems’ reliability. As it can be seen from the results in Table VI and Figure 3, 42.98% of the participants' responses fall in the range of agree and strongly agree which shows that the participants are moderately satisfied with the system’s efficiency. It is worth mentioning that 23.14% of the participants are neutral on their response and 33% consider the CMS unreliable and give feedback as disagree and strongly disagree. The impact on the control of the system upon the freedom of use was examined and the results are shown in Table V. The minimum and maximum scores were 50 and 126 and the mean was 84 which is near to minimum. As it can be seen from the results in Table VI and Figure 4, only 37% of the participants' responses fall in the range of agree and strongly agree which shows that the users were unsatisfied with the usability features. It is worth mentioning that 13% of the participants are neutral on their response on that feature and 49.2% of them consider that the CMS is not easy to use and does not provide easy learning and gave feedback as disagree and strongly disagree. Finally, the capability of the system to provide efficient usage was examined and the results are shown in Table V. The minimum and maximum scores were 17 and 62 and the mean was 42. By this it can be concluded that the participants were moderately satisfied with the efficiency of the CMS. Table VI and Figure 5 present the details of how the system efficiency responds to the user. As it can be seen from the results in Table VI, 45.98% of the participants’ responses fall in the range of agree and strongly agree which shows that the participants are moderately satisfied with the system’s efficiency, 25.23% of the participants are neutral on their response and only 29% of them consider the CMS as not efficient and give feedback as disagree and strongly disagree. Fig. 4. Percentage distribution of CMS usability characteristics feedback Fig. 5. Percentage distribution of CMS efficiency characteristics feedback 17 15 14 8 47 57 36 20 18 15 25 42 17 13 21 27 6 5 8 8 0 20 40 60 80 100 120 Suitab ility Sec urity Accurateness Interoperability F u n c ti o n a li ty Strongly Agree Agree May be Disagree Strongly Disagree 19 9 7 45 30 23 13 38 24 22 22 27 6 6 23 0 20 40 60 80 100 120 Maturity Rec overability Fault tolerance R e li a b il it y Strongly Agree Agree May be Disagree Strongly Disagree 8 12 15 11 32 29 26 20 17 14 14 11 28 34 31 33 20 15 18 28 0 20 40 60 80 100 120 Understandability Learnability Operability Attractiveness U sa b il it y Strongly Agr ee Agree May be Disagree Strongly Disagree 18 15 28 34 24 29 24 20 10 7 0 20 40 60 80 100 120 Time behav ior Resource utilization E ff ic ie n c y Strongly Agr ee Agree May be Disagree Strongly Disagree Engineering, Technology & Applied Science Research Vol. 9, No. 6, 2019, 5037-5040 5040 www.etasr.com Hafeez et al.: A Survey on the Adaption of CMS in Pakistani Universities TABLE VI. FREQUENCY STATISTICS Functionality Sub-characteristics Questions Strongly Agree Agree Maybe Disagree Strongly Disagree N % N % N % N % N % Suitability Can the CMS do the required tasks? 17 16.19 47 44.76 18 17.1 17 16.19 6 5.71 Security Does the CMS prevent unauthorized access? 15 14.29 57 54.29 15 14.3 13 12.38 5 4.76 Accuracy Does the CMS give results as expected? 14 13.33 36 34.29 25 23.8 21 20 8 7.62 Interoperability Does the CMS interact with another system? 8 7.62 20 19.05 42 40 27 25.71 8 7.62 Reliability Maturity Has a maximum number of errors in the CMS been removed over time? 19 18.1 45 42.86 13 12.38 22 20.95 6 5.71 Recoverability After failure, can the CMS continue working and recover lost data? 9 8.57 30 28.57 38 36.19 22 20.95 6 5.71 Fault tolerance Is the CMS able to handle errors? 7 6.67 23 21.9 24 22.86 27 25.71 23 21.9 Usability Understandability Does the user apprehend how to easily use the system? 8 7.62 32 30.48 17 16.19 28 26.67 20 19.1 Learnability Can the end-user learn the system easily? 12 11.4 29 27.62 14 13.33 34 32.38 15 14.3 Operability Can the user use the system without much effort? 15 14.3 26 24.76 14 13.33 31 29.52 18 17.1 Attractiveness Does the system has a good look and feel? 11 10.5 20 19.05 11 10.48 33 31.43 28 26.7 Efficiency Time behavior How rapidly does the system respond and answer to queries? 18 17.1 28 26.67 24 22.86 24 22.86 10 9.52 Resource utilization Does the system efficiently utilize system resources? 15 14.3 34 32.38 29 27.62 20 19.05 7 6.67 III. CONCLUSION AND RECOMMENDATIONS This paper presented an evaluation of CMS in four dimensions: functionality, reliability, usability, and efficiency. These characteristics are a major part of ISO/IEC 9126-1. From the evaluation, it was observed that there were no issues related to the functionality of the system regarding suitability, security, accuracy, and interoperability, with 51.5% of the participants’ responses falling in the range of agree and strongly agree. In addition, the results show that participants were moderately satisfied with the reliability of the system regarding maturity, recoverability, and fault tolerance, while around 43% of the participants agree and strongly agree with the reliability features of the CMS. The participants were not very much satisfied from the usability features of the CMS and only 37% of them agreed or strongly agreed with the effectiveness of usability features such as understandability, learnability, operability, and attractiveness. Finally, it was observed that the participants were also moderately satisfied with the efficiency and around 43% of them agreed and strongly agreed with the questions regarding the efficiency features. As a conclusion, it is observed that most participants are satisfied with the CMS quality characteristics, but it would be better to enhance its usability features. REFERENCES [1] M. Hemmati, H. Hosseini, “Effect of IT application on project performance focusing on the mediating role of organizational innovation, knowledge management and organizational capabilities”, Engineering, Technology & Applied Science Research, Vol. 6, No.6, pp. 1221-1226, 2016 [2] M. Ben Ayed, L. Zouari, M. Abid, “Software in the loop simulation for robot Manipulators”, Engineering, Technology & Applied Science Research,Vol. 7, No. 5, pp. 2017-2021, 2017 [3] F. Idachaba, E. Idachaba, “Robust e-health communication architecture for rural communities in developing countries”, Engineering, Technology & Applied Science Research, Vol. 2, No. 3, pp. 237-240, 2012 [4] M. Odhiambo, P Umenne, “NET-Computer: Internet computer architecture and its application in e-commerce”, Engineering, Technology & Applied Science Research, Vol. 2, No. 6, pp. 302-309, 2012 [5] Z. Unal, A. Unal, “Evaluating and comparing the usability of web-based course management systems”, Journal of Information Technology Education, Vol. 10, pp. 19-38, 2011 [6] G. Sabau, M. Munten, A. R. Bologa, R. Bologa, T. Surcel, “An evaluation framework for higher education ERP Systems”, WSEAS Transactions on Computers, Vol. 8, No. 11, pp. 1790-1799, 2009 [7] A. Rainer, G. Auth, “Campus-management-system”, Business & Information Systems Engineering, Vol. 52, No. 3, pp. 185-188, 2010 [8] A. S. Sife, E. T. Lwoga, C. Sanga, “New technologies for teaching and learning: Challenges for higher learning institutions in developing countries”, International Journal of Education and Development Using Information and Communication Technology, Vol. 3, No. 2, pp. 57-67 2007 [9] Higher Education Pakistan, HEC vision 2025, available at: https://www.hec.gov.pk/english/HECAnnouncements/Documents/Anno uncement/HEC-Vision-2025.pdf, accessed September 28, 2018 [10] H. W. Jung, S. G. Kim, C. S. Chung, “Measuring software product quality: A survey of ISO/IEC 9126”, IEEE Software, Vol. 21, No. 5, pp. 88-92, 2004 [11] P. Botella, X. Burgues, J. P. Carvallo, X. Franch, G. Grau, J. Marco, C. Quer, “ISO/IEC 9126 in practice: what do we need to know”, Software Measurement European Forum, 2004