103 |JISAE (Journal of Indonesian Student Assessment and Evaluation)|Volume 7 Number 2 https://doi.org/10.21009/JISAE JISAE (Journal of Indonesian Student Assessment and Evaluation) ISSN : P-ISSN: 2442-4919│E-ISSN: 2597-8934 Vol 7 No 2 (2021) Website : http://journal.unj.ac.id/unj/index.php/jisae THE EVALUATION OF IMPLEMENTING ONLINE LEARNING AT UNIMED FACULTY OF ENGINEERING Zulkifli Matondang1 Universitas Negeri Medan Sarwa2 Universitas Negeri Medan Liana Atika3 Universitas Negeri Medan Sutrisno4 Universitas Negeri Medan ABSTRACT This research aims to evaluate the online learning implementation activities at the UNIMED Faculty of Engineering to determine the various weaknesses and constraints. The research approach used is the CIPP model of Stufflebeam. And it focused on the content/material, facilities, processes, and outputs of online learning. Data were obtained from UNIMED Faculty of Engineering students through online questionnaires. The results showed that 1) the availability of learning resources is still low, as a content aspect, 2) network strength is weak at the input aspect, 3) the helpdesk service is lacking at the process aspect, and 4) the online effectiveness is low at the product aspect. On average, out of these four components, the lowest is online learning tools. In conclusion, this research found that learning facilities contributed to online learning effectiveness at the UNIMED Faculty of Engineering. Keywords: Evaluation, CIPP Model, Online Learning Address for Correspondence: 1zulkiflimato@gmail.com; 2sarwa@unimed.ac.id; 3liana_atika@unimed.ac.id; 4sutrisno_unimed@yahoo.com INTRODUCTION The development of information and communication technology (ICT) led to the switch from offline to online to learning by the UNIMED Faculty of Engineering (EF) lecturers. They creatively and innovatively implemented this academic process by planning and arranging learning activities through existing networks or Online Learning Models (OLM). At the beginning of 2020, the whole world, including Medan City in Indonesia, contracted a contagious disease called covid-19, which led to the mandatory conduction of learning activities online. Therefore, due to the pandemic, the rector implemented the following online lecturing processes in accordance with Letter No. 000809/UN.33/SE/2020, namely Theory (Face-to-Face, Structured Tasks, and Independent Tasks) are to be carried out using E-Learning, Vi-Leaning, Mailing List, WA Group, Line, Skype, or similar methods. Practical lectures (Laboratory, Workshop, Studio, Studio, Microteaching) are carried out by giving other relevant assignments by the course achievements. Eko Kurtanto stated that online learning is quite effective in improving student learning outcomes (Kuntarto, 2017). Its application creates a more active, innovative, creative, effective, and less boring atmosphere, which increases student motivation and ultimately raises achievement. For online learning to run effectively, every element http://journal.unj.ac.id/unj/index.php/jisae mailto:zulkiflimato@gmail.com mailto:sarwa@unimed.ac.id mailto:liana_atika@unimed.ac.id mailto:sutrisno_unimed@yahoo.com 104 involved needs to meet the requirements. The lecturer in implementing online learners needs to first possess the knowledge and skills in using various applications. One of the applications used is SIPDA. This application is designed for online learning forinternal university. Apart from SIPDA, there are several paid and free applications used. One of the factor that supports the implementation of online learning is shareable lecture material through the network in the form of softcopy. The students need to possess adequate knowledge and skills of online learning activities, with facilities (computer or gadget) that are connected to the internet network to enable them to learn through the network (online). The observations of the online learning implementation at UNIMED Faculty of Engineering are varied. Llecturers are expected to carry out online learning on each subject using their perceptions, conditions, and abilities. This led to the inadequate implementation of online learning. There fore, to determine the quality of online learning and to reveal its implications, an in-depth study was conducted. According to Sitompul, one of the important components in planning a program is to state the general and specific goals in advance (Sitompul et al., 2018). Several models are used in the program evaluation activities, one of which is the CIPP. This model was applied to evaluate the implementation of online learning at EF UNIMED. Stuffle beam developed the CIPP model, and it is an acronym for the wordsContext, Input, Process, and Product. It includes four types of decisions, namely (1) planning, which affect the choice of objectives, (2) structuring, which determines the optimal strategy and procedural design for achieving the goals determined from the planning phase, (3) implementing, which include implementation and improvement of the design, method or strategy that has been selected, and (4) recycling, which determine whether the online learning program is repeated, changed or discontinued. It is important to evaluate online learning at FT Unimed to find out various obstacles and can be make decisions to fix them. By improving the process and means of online learning, it will improve the quality of FT Unimed graduates. The CIPP model is very suitable to be used to evaluate a program such as online learning. The CIPP model describes various weaknesses that occur in online learning at FT Unimed, from the aspect of content (learning planning), input (learning facilities), process (learning implementation), and product (learning outcomes). Evaluation of online learning at FT Unimed has never been done, let alone using the CIPP model approach. Matondang stated that the purpose of program evaluation is to obtain accurate and objective information (Matondang & Sitompul, 2017). Therefore, the evaluation is intended to determine the effectiveness and level of achievement (progress/success rate) of an activity, especially the online learning program at EF UNIMED. The term evaluation is often equated with assessment, although several other terms such as scoring and judgment are sometimes translated as assessment (Norman E, 1990). METHOD This is an evaluative research conducted at the EF of UNIMED, from May to November 2020, using the CIPP model designed by Stufellbeam. The CIPP model is an evaluation activity and an acronym for the words content, input, process, and product. Furthermore, this study utilised qualitative and quantitative data. Qualitative data is in the form of information on the process and implementation of online learning experienced by EF UNIMED students, such as the preparation process, work on assignments, implementation of video conferences, and assessment systems in online learning. Quantitative data is in the form of descriptive information on the 105 |JISAE (Journal of Indonesian Student Assessment and Evaluation)|Volume 7 Number 2 quality of online learning implementation, the achievement of student competencies according to the subjects, and the effectiveness of online learning implementation. The data sources are the students and lecturers involved in implementing online learning. Several techniques and collection tools were used to obtain data. The data collection techniques weresurveys, observations, interviews, and online questionnaires. While the tools used are the assessment sheet, documentation check, and an online questionnaire. The data analysis model is in accordance with Lodico, Spaulding, & Voegtle concepts (Putra, 2012). Itcomprises of the following stages, namely 1) organizing and carefully checking the data, 2) re-checking the data, 3) carrying out further data processing by summarising, describing, and organising, codes containing categories that are more specific and differentiated from others, and 4) carry out the final analysis, make interpretations, and conclusions containing the results of the research. RESULTS AND DISCUSSION Results The respondents in this study were 882 UNIMED Faculty of Engineering students. The distribution of the proportion of students based on the existing majors is shown in Figure 1. Figure 1. Proportion of Student Respondents based on Departments. Figure 1 shows that 453 (51%) students majoring in FWE provided data online, while the least was 87 (10%) students from MEE majors. The data analysis results provided by UNIMED Faculty of Engineering students for online planning documents made by lecturers at SIPDA obtained an average score of 2.94 with a standard deviation of 0.73. Table 1 shows the average score of student responses on the learning planning document. Table 1. The Average of Student Responses About Learning Planning. No Aspects Average Score Deviation 1 Subject Identity 3.31 0.66 2 Clarity of competency formulation 3.02 0.76 3 Achievement of learning objectives 2.84 0.76 4 Outline of material description 3.09 0.71 5 Learning media used 2.94 0.73 6 Online learning activities 2.71 0.83 7 Clarity of content being worked on 2.80 0.76 8 Specific assessment instruments 2.96 0.67 9 Completeness of assessment instruments 2.97 0.69 125; 14% 87; 10% 217; 25% 453; 51% Building Engineering Education (BEE) Mechanical Engineering Education (MEE) Electrical Engineering Education (EEE) Family Welfare Education (FWE) 106 10 Availability of learning resouurces 2.74 0.75 Average 2.94 0.73 This data shows that lecturers' quality of online learning planning at SIPDA is in the good category with an average of 2.94. The assessed smallest aspect of 10 was found in online learning activities, and the availability of learning resources was the lowest, with an average of 2.71 and 2.74, respectively. A good aspect is the identity and outline of the material description of the planning that the lecturer supports in SIPDA. These results indicate that online learning activities at the UNIMED Faculty of Engineering need to be improved. Furthermore, efforts are made to improve teachers' and students' competence. The data analysis results regarding the implementation of online learning in terms of the available facilities and based on students' conditions and feelings, are shown in Table 2. Table 2. The Average of Student Responses About Online Learning Facilities. No Aspects Average Score Deviation 1 Availability of online devices 2.68 0.70 2 Online network strength 2.31 0.74 3 Student competence in online learning implementation 2.84 0.76 4 Completeness of uploaded material 2.82 0.70 5 Completeness of LMS features on SIPDA 2.78 0.73 6 Ease of applicating SIPDA 2.69 0.81 7 Availability of SIPDA guidelines 2.72 0.80 Average 2.69 0.75 Table 2 shows that the weakest aspect of online learning facilities is the network strength and availability of devices with mean values of 2.31 and 2.68, respectively. The main requirement for online learning is the existence of good and stable internet. Furthermore, the quality of online learning can be increased through the provision of networks and devices. In terms of student competence, the feature aspects of SIPDA and its application are in a good category. This shows that generally students are able to adapt and participate in online learning which is carried out at UNIMED Faculty of Engineering. The survey results on the online implementation process are shown in Figure 2. Figure 2. The Average of StudentResponses to theOnline Learning Process Figure 2 shows that the aspects of helpdesk and communication processes on online learning obtained average values of 2.83, and2,91, respectively. The research 2,91 3,18 3,07 2,96 3,11 3,13 2,91 3,04 2,83 3,02 2,6 2,7 2,8 2,9 3 3,1 3,2 3,3 Online accuracy on schedule Online presence of students Online student participation Variance of features used in SIPDA Completeness of assignments online On time submission of assignments Online communication process Online assignment appraisal process Service process (helpdest) on online Average 107 |JISAE (Journal of Indonesian Student Assessment and Evaluation)|Volume 7 Number 2 data obtained indicate that both service processes need to be improved. Furthermore, the accuracy of online learning according to the specified schedule also needs to be improved. The data analysis results on the output of online learning are shown in Table 3. Table 3. The Average of Student Responses on Learning Planning No Aspects Average Score Deviation 1 Achievement of learning targets 2.86 0.64 2 Adequacy of time according to material 2.84 0.68 3 Number of tasks in online learning 2.77 0.75 4 Responses to online learning 2.77 0.76 5 Achievement of online competences 2.66 0.73 6 Effectiveness of learning through online 2.53 0.76 Average 2.74 0.72 Table 3 shows that learning online's effectiveness and competency outcomes have average values of 2.53 and 2.66, respectively. These results indicate that the effectiveness of learning and competency outcomes through online learning is in a low category. Therefore, for the lecturers to improve online learning, the effectiveness and achievement of competencies need to be increased. The data analysis results obtained from the respondents show that the product of online learning activities has an average of 2.74 with a deviation of 0.72. The four components evaluated, namely content, input, processes, and products, are shown in Figure 3. Figure 3. The Average of Student Responses to Online Learning Components Figure 3 shows that the facilities components' average score has the lowest score of 2.69, followed by the product component at 2.74. These results indicate that the implementation of online learning needs to be improved, especially in structuring and improving learning facilities. Discussion The evaluation research results on the implementation of online learning at Engineering Faculty at UNIMED indicate that the competence of learning planning (content), and resources, had the lowest average value compared to other aspects. 0 0,5 1 1,5 2 2,5 3 3,5 Online Planning Online Tools Online Process Online Products 2,94 2,69 3,02 2,74 0,73 0,75 0,71 0,72 Average S Deviation 108 These results indicate that a lecturer needs to compile an online learning plan capable of being implemented properly. Another thing that needs to be improved and added is learning resources usable in online learning. Therefore, in increasing the quality of online learning planning, it is necessary to have a common perception of the implementation process for the objectives to be achieved according to plan. This is in line with Eko Kurtanto's opinion, which stated that online learning's success is carried out by increasing teacher competence in making good lesson plans (Kuntarto, 2017). Evaluation activities on the input component need to be in accordance with the facilities from the online learning program. The evaluation show that the aspects of network strength and the availability of online learning tools are of low quality. The internet network and the availability of devices are very important for online learning success at the implementation stage. Furthermore, in improving the quality of online learning, it is necessary to increase the strength and breadth of the internet network and fulfill online learning tools. Evaluation is in terms of various aspects related to the online learning process. The evaluation results show that the helpdesk and the online communication processes are of the lowest quality. The implementation of online learning is still relatively new, therefore students, staff, and lecturers still need guidance and training to run effectively. To improve the online learning process at, employees, or helpdesk need to be enhanced. Technically, the helpdesk should always be willing to assist students or lecturers in the proper running of the system. Similarly, lecturers need to enhance their services to provide quality communication in online learning, which is important in learning both orally and in writing. Evaluation activities on product components are reviewed from online learning outputs at EF UNIMED. The evaluation results on the product components showed that the effectiveness of online learning and student competencies' achievement had low quality. This evaluation indicates that a method or strategy is needed for online learning activities to be implemented more effectively both in terms of implementation time and the achievement of learning objectives. Based on these evaluation results, it is necessary to carry out a more appropriate learning strategy or method through online learning. The overall online learning activities are components with lower quality. Therefore it is necessary to review the implementation process. According to Sitompul, the implication of implementing the CIPP model evaluation is that it affects the decisions making process, methods, or strategies carried out in EF UNIMED online learning (Sitompul et al., 2018). CONCLUSION Based on the results and analysis of the research data, the following conclusions were drawn: 1) For the content component evaluated in online learning, the aspect of availability of learning resources is still low with a mean score of 2.74, 2) At the input component, the lowest aspect is the power of the online network, with a mean score of 2.64, 3) At the process component, the helpdesk service aspect for online learning still lacks with an average score of 2.83, and 4) Product components considered lacking are the effectiveness aspects of online learning with an average score of 2.53. Of the four components, the lowest is online learning tools. Therefore, to improve the quality of online learning the following need to be achieved 1) additional learning resources provided by lecturers, 2) additional facilities and strong networks to support online learning, 3) enhance the quality of services provided by helpdesk for 109 |JISAE (Journal of Indonesian Student Assessment and Evaluation)|Volume 7 Number 2 learning online, and 4) increase the effectiveness and achievement of student competencies in online learning. Acknowledgment The authors are grateful to the UNIMED Rector for funding this research REFERENCE Kuntarto, E. (2017). Keefektifan Model Pembelajaran Daring dalam Perkuliahan Bahasa Indonesia di Perguruan Tinggi. Journal Indonesian Language Education and Literature, 3(1), 53–65. https://www.syekhnurjati.ac.id/jurnal/index.php/jeill/article/view/1820 Matondang, Z., & Sitompul, H. (2017). Evaluation of Learning Outcomes. Al-Hayat Foundation. Norman E, G. (1990). Measurement and Evaluation in Teaching. McMillan Inc. Putra, N. (2012). Educational Qualitative Research Methods. Raja Grafindo Persada. Sitompul, H., Mursid, R., & Matondang, Z. (2018). Evaluation of Learning Programs. Open University Publisher.