Available online at www.HighTechJournal.org HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 233 ISSN: 2723-9535 Utilization of the Weighted Product-Based CIPP Evaluation Model in Determining the Best Online Platform Dewa Gede Hendra Divayana 1* , P. Wayan Arta Suyasa 1 , Nyoman Santiyadnya 2 , Made Susi Lissia Andayani 1 , I Made Sundayana 3 , I Nengah Dasi Astawa 4 , Ni Wayan Rena Mariani 5 , Gusti Ayu Dessy Sugiharni 5 1 Department of IT Education, Universitas Pendidikan Ganesha, Singaraja, Bali, 81116, Indonesia. 2 Department of Electrical Education, Universitas Pendidikan Ganesha, Singaraja, Bali, 81116, Indonesia. 3Department of Health, Sekolah Tinggi Ilmu Kesehatan Buleleng, Singaraja, Bali, 81171, Indonesia. 4Department of Management, Universitas Pendidikan Nasional, Denpasar, Bali, 80224, Indonesia. 5 Department of Digital Business and Entrepreneurship, Institut Pariwisata dan Bisnis Internasional, Denpasar, Bali, 80239, Indonesia. Received 09 December 2022; Revised 17 February 2023; Accepted 26 February 2023; Published 01 March 2023 Abstract Since the COVID-19 pandemic, there have been many free online platforms that can be used to support the online learning process at health colleges in Bali. However, it is difficult to determine the best online platform from the various choices of free online platforms that are scattered on the internet. Therefore, it needs innovations that contribute to helping solve these problems. One model as an innovation that can be used and contributes to solving problems is the Weighted Product-based CIPP evaluation model. The model needs to be measured for the quality of its calculations to ensure success in determining the best online platform. Therefore, this research aimed to show the quality of the Weighted Product method calculation integrated with the CIPP (Context-Input-Process-Product) model in determining the best platform used in health colleges during the COVID-19 pandemic. The instrument used to assess the quality of that calculation was a questionnaire consisting of eight questions. The subjects involved in the assessment were 20 experts. The research was carried out at several health colleges in Bali. The analytical technique used in analyzing the research data was descriptive-quantitative. The analysis was carried out by comparing the quality percentage of the calculation simulation with the quality standard based on an eleven-point scale. The study results showed the quality percentage of calculation simulation was 87.250%, so it was included in the very good category. This research has a significant impact on the progress of the educational evaluation field through research findings in the form of the appearance of the combination of the Weighted Product method with the CIPP evaluation model. The novelty of this research is the combination of the Weighted Product method and the CIPP model, which makes it easier for educational evaluators to determine the best online platform that supports online learning during the COVID-19 pandemic and even post-COVID-19. Keywords: Weighted Product; COVID-19 Pandemic; CIPP; Online Platform; Online Learning. 1. Introduction Online learning during the COVID-19 pandemic is the most suitable strategy to use to minimize the spread of the coronavirus in college environments. Many online platforms can be used to realize online learning. Some of those * Corresponding author: hendra.divayana@undiksha.ac.id http://dx.doi.org/10.28991/HIJ-2023-04-01-015  This is an open access article under the CC-BY license (https://creativecommons.org/licenses/by/4.0/). © Authors retain all copyrights. https://creativecommons.org/licenses/by/4.0/ https://orcid.org/0000-0002-6702-3253 https://orcid.org/0000-0001-7264-2529 https://orcid.org/0000-0002-5439-2340 https://orcid.org/0000-0001-9676-8732 https://orcid.org/0000-0002-3109-3014 https://orcid.org/0009-0000-7238-0282 https://orcid.org/0000-0003-4397-8044 https://orcid.org/0000-0003-2578-0456 HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 234 platforms include Kelase, Schoology, Moodle, SEVIMA EdLink, Edmodo, Quipper School, etc. [1-4]. However, reality showed that not all of those platforms were able to effectively make the learning process run well. This also occurs specifically in several health colleges in Bali. At several health colleges in Bali, the use of online platforms is only used to upload materials, transfer materials, upload assignments, and answer exams. The assessment process is also limited to an assessment in the cognitive domain, even though, in reality, an attitude and psychomotor assessment are also very much needed. Therefore, it is necessary to conduct a comprehensive evaluation to determine the best platform that can be used in online learning, especially in health colleges. The evaluation carried out should combine evaluation components in the field of education with decision-support methods in the field of computers so that the evaluation results become more accurate. Based on those needs, new innovations are needed to realize comprehensive evaluation activities. The innovation can be in the form of utilizing the CIPP evaluation model integrated with the Weighted Product method. Referring to that innovation, the purpose of this study was to show the use of the Weighted Product method combined with the CIPP evaluation model in determining the best online platform used in health colleges during the COVID-19 pandemic. The research problem was, “How to calculate the Weighted Product method combined with the CIPP model to determine the best online platform used in health colleges during the COVID-19 pandemic?” Several previous research results that baseline this research include Purwaningsih and Dardjito’s research [5], which showed the use of the CIPP evaluation model to evaluate the effectiveness of e-learning during the COVID-19 pandemic. The limitation of Purwaningsih and Dardjito’s research was that it had not shown a combination of decision support system methods with educational evaluation models to obtain accurate evaluation results of the e-learning platform suitable implemented during the COVID-19 pandemic. Damayanti et al.’s research [6] only showed the CIPP model used to evaluate the effectiveness of online learning in universities. Damayanti et al. had not implemented a decision support method combined with the CIPP model in determining the best platform for supporting the effectiveness of online learning. Anh & Pang’s [7] showed the application of the CIPP model to evaluate the implementation of online-based English language teaching. The limitation of Anh & Pang’s research was that it had not shown the best online platform that was able to be used to support English language teaching. Prayogo et al. [8] focused on determining the evaluation results of the implementation of online-based distance learning, which refers to the CIPP evaluation component. Prayogo’s et al. research had not shown any combination of the CIPP model with decision support methods in determining the best online platform that supports distance learning. Research by DeCoito & Estaiteyeh [9] showed the use of the CIPP model in evaluating the curriculum and assessment of science/STEM teachers in online learning in Canada during the COVID-19 pandemic. The limitation of DeCoito & Estaiteyeh's research was that it did not show the best online platform that supports the implementation of a quality curriculum and assessment in online learning during the COVID-19 pandemic. Therefore, in DeCoito & Estaiteyeh’s research, a combination of decision support methods and the CIPP model is needed to perform accurate calculations in determining the best online platform to support learning during the COVID-19 pandemic. Toan et al. [10] showed the use of decision support methods in assessing and selecting the best e-learning platform to support the learning process. The limitation of Toan et al.’s research was that it had not shown an educational evaluation model integrated with the decision-making method used in the research, so the platform chosen had not been able to comprehensively facilitate the needs of the learning process in the field. Ong et al. [11] showed an analysis of the accuracy of selecting online learning attributes by students during the COVID-19 pandemic. The limitation of Ong et al.’s research was that it did not show an accurate calculation process in determining the selection of online learning attributes assisted by decision support methods and educational evaluation models. Nguyen & Nguyen [12] showed that the Schoology platform is suitable for improving student learning abilities. The limitation of Nguyen and Nguyen’s research was that it did not show detailed calculation processes in determining the choice of the Schoology platform as the best platform. Research by Shashiprabha et al. [13] showed the results of an analysis of e-learning platforms, but the best platform that can be used to support e-learning has not yet appeared. Cabual & Cabual [14] showed the students challenges in implementing learning using online platforms during COVID-19. The limitation of the Cabual & Cabual’s research was that it had not been shown what the best platform certainty was for the learning process during COVID- 19. Based on some of the limitations of those previous studies, the idea of this research is very appropriate to be expressed to overcome the limitations of previous research related to selecting the best platform for online learning. The idea of this research is to show the calculation of the Weighted Product method combined with the CIPP model to determine the best platform to use in supporting online learning during the COVID-19 pandemic and even post-COVID-19. HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 235 2. Method 2.1. Research Approach The approach of this research was development with a focus on the calculation simulation of the Weighted Product integrated with the CIPP model and the quality assessment of the calculation simulation of the Weighted Product method integrated with the CIPP model. The Weighted Product calculation simulation is more focused on the correctness of the process and the quality stages of the calculation of the three formulas in the Weighted Product, while the quality assessment is focused on the validity of the simulation results of the calculation of the Weighted Product integrated with the CIPP model. The simulation stages for calculating the Weighted Product in this study can be seen in Figure 1. The stages for evaluating the simulation quality for the calculation of the Weighted Product method integrated with the CIPP model can be seen in Figure 2. Figure 1. The Simulation Stages for Calculating the Weighted Product Figure 2. The Stages of Quality Assessment of Simulation Calculation of the Weighted Product Method integrated with the CIPP Model Figure 1 shows the five stages that must be passed in the Weighted Product calculation simulation. Stage-1 is the determination of the initial data for simulation. Initial data for simulation were obtained from the average of interest rating scores given by respondents to each aspect of the CIPP evaluation model. Stage-2 is the revision of weights by experts for each CIPP evaluation component. Stage-3 is the calculation of the S vector using the formula shown in Determination of the Initial Data for the Simulation Revision of Weights from Experts for Each Evaluation Component Calculations of S Vector Ranking for Determining the Best Online Platform Calculations of V Vector Giving Assessment Questionnaires to Experts Assessment Process by Experts Results of Experts’ Assessment Categorization of Calculation Simulation Quality Cross Check of Expert Assessment Results with Quality Standards of Eleven Scales HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 236 Equation 2. Stage-4 is the calculation of the V vector using the formula shown in Equation 3. Stage-5 is ranking to determine the best online platform based on the highest V-vector score. Figure 2 shows the five stages in assessing the simulation quality of the Weighted Product calculation combined with the CIPP model. Stage-1 is giving assessment questionnaires to experts. The questionnaires are used as a tool to obtain an assessment score from experts on the quality of calculation simulations. Stage-2 is the assessment process carried out by experts. Stage-3 is the activity of regularly collecting and compiling all the scores that have been obtained from the results of expert’s assessment. Stage-4 is an activity to cross-check between the score from the expert’s assessment and the quality standard of the calculation simulation which refers to eleven’s scale. Stage-5 is the categorization of the quality of the calculation simulation process by reference to the quality standards of the eleven’s scale. 2.2. Simulation Formula There are three formulas for simulating the calculation of the Weighted Product method. The first formula for the weighting improvement process. The first formula is shown in Equation 1 [15–18]. The second formula is to determine the S vector. The second formula is shown in Equation 2 [19–23]. The third formula for determining the V vector. The third formula is shown in Equation 3 [24–32]. 𝑤𝑗 = 𝑤𝑗 ∑ 𝑤𝑗 (1) 𝑆𝑖 = ∏ 𝑥𝑖𝑗 𝑤𝑗𝑛 𝑗=1 𝑤ℎ𝑒𝑟𝑒: 𝑖 = 1,2, … , 𝑚 (2) 𝑆 is the criteria preference which is often called the 𝑆 vector. 𝑥 is the criterion value. 𝑤𝑗 is a positive weight for the profit attribute and a negative weight for the cost attribute. wj must be valuable of 1. 𝑉𝑖 = 𝑆𝑖 ∑ 𝑆 𝑤ℎ𝑒𝑟𝑒: 𝑖 = 1,2, … , 𝑛 (3) 𝑉 is an alternative preference for determining rank. This is often called a 𝑉 vector. 2.3. Subject, Object, and Location of Research The subjects involved in the quality assessment of the simulation results of Weighted Product calculations were 20 experts. The 20 experts consisted of 10 education experts and 10 informatics experts. The subjects involved in the weight improvement process were six experts, consisting of three education experts and three informatics experts. Determination of subjects for this research was carried out based on the purposive sampling technique. The reason for using this technique is because if you choose another sampling technique, it will be difficult to determine a subject that is truly sensitive and understands in depth about the online platform used in learning. In general, this purposive sampling technique makes it easier for researchers to obtain data from sources that are indeed appropriate and have in-depth experience with the object under research. All subjects involved in this research had in-depth knowledge and experience of the role of online platforms in supporting learning. The advantage of using this purposive sampling technique is that it increases the sensitivity of the assessments made by the subjects involved in this research, because the subjects will provide an assessment according to their experience. The object of this research was a Weighted Product method combined with the CIPP model to determine the best online platform. The object of this research was the research focus because it was based on ideas raised to overcome problems found in the field related to difficulties in determining the best online platform to support the online learning process. The research was conducted at several health universities in Bali. The universities are located in several regencies, including: Tabanan, Badung, Denpasar, Klungkung, Buleleng, and Gianyar. The reason for selecting several health colleges as research locations was to obtain differences in the characteristics of online platform users. The existence of differences in the characteristics of online platform users is very good, because it will provide a more objective sensitivity to the assessment results and a variety of perspectives on the online platform being assessed. 2.4. Data Collection Instrument The instrument used to assess the quality of the calculation simulation was a questionnaire consisting of eight questions. Question-1 about the initial data conditions for the simulation. Question-2 about the results of the weight improvement from the expert. Question-3 about the accuracy of the calculation results for the S vector. Question-4 about the accuracy of the V vector calculation results. Question-5 about the accuracy of the ranking results in the context component. Question-6 about the accuracy of the ranking results in the input component. Question-7 about the accuracy of the ranking results in the process component. Question-8 about the accuracy of the ranking results in the product component. HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 237 2.5. Data Analysis Technique The results of the analysis of the calculation quality assessment using the quantitative descriptive technique. This analysis technique was carried out by comparing the quality of the calculation simulation results with quality standards that refer to the eleven’s scale. The formula for calculating the quality percentage is shown in Equation 4 [33–39], while the quality standard, which refers to the eleven’s scale, is shown in Table 1 [40–44]. 𝑃 = 𝑓 𝑁 × 100% (4) where 𝑃 is percentage of quality, 𝑓 is total of the acquisition value, and 𝑁 is total of maximum value. Table 1 shows the eleven quality standard scales used as the basis for categorizing the quality of the calculation simulation of the Weighted Product method, which is integrated with the CIPP model. If the quality percentage range is 75%–100%, then the quality of the average calculation simulation is good, so there is no need to repeat the calculation. If the range of quality percentages is less than 75%, then the quality of the calculation simulation is generally classified as poor, so a re-simulation is necessary. Table 1. Quality Standards Based on Eleven’s Scale Classification of Quality Range of Quality Percentage Excellent 95 to 100 Very Good 85 to 94 Good 75 to 84 More than Enough 65 to 74 Enough 55 to 64 Almost Enough 45 to 54 Minus 35 to 44 Very Minus 25 to 34 Poor 15 to 24 Very Poor 5 to 14 Highly Poor 0 to 4 3. Results and Discussion 3.1. Online Platforms used in Health Colleges in Bali Several online platforms used at health colleges in Bali to support the online learning process during the COVID-19 pandemic, including: Microsoft Teams, Kelase, Moodle, and SEVIMA EdLink. The display of some of these platforms can be seen in Figure 3 to 6. Figure 3. Display of Microsoft Teams HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 238 Figure 4. Display of Moodle Figure 5. Display of Kelase Figure 6. Display of SEVIMA EdLink HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 239 Microsoft Teams is a modern application offered by Microsoft. This application is a hub for a team, both in small or large-scale organizations that allow users to collaborate and communicate easily whenever and wherever they are. Microsoft Teams can be accessed through this URL: https://www.microsoft.com/en/microsoft-teams/group-chat- software. Moodle is a web-based service that assists in online learning activities. Moodle is an acronym for Modular Object- Oriented Dynamic Learning Environment which can be said to be a dynamic learning place using models and object- oriented. Moodle can be accessed through this URL: https://moodle.org/. The Kelase application is an application developed by PT. Edukasi Satu Nol Satu from Indonesia helps education organizations provide online services so they can collaborate, learn, and exchange knowledge with various features and ease of access. Kelase can be accessed through this URL: https://www.kelase.com/. SEVIMA EdLink is an online learning platform made by Indonesians which has several facilities, including online presence, remote video conferencing, notifications of online lecture schedules, interactive quizzes with attractive packaging, discussion forums for material that is easy but still interactive, and recapitulation of each student's presence. SEVIMA EdLink can be accessed through this URL: https://edlink.id/. 3.2. Calculation Simulation of Weighted Product Method Based on those online platforms, it was necessary to determine the best platform that was able to be used in the learning process during the COVID-19 Pandemic. Therefore, in this research, calculation simulation was carried out to determine the best online platform using the CIPP model based on Weighted Product. The calculation simulation process can be shown as follows. 1) Determination of Initial Data for Simulation The initial data used for the calculation simulation of the Weighted Product method consists of the average score of the interest rating given by the respondents to each CIPP evaluation aspect. The respondents involved were 10 experts. The initial data intended can be seen in Table 2. Table 2. Initial Data for Weighted Product Calculation Simulation Evaluation Aspects Platforms Evaluation Components Context Input Process Product Vision and mission of organizing online learning Microsoft Teams 3.90 1.00 1.00 1.00 Kelase 3.80 1.00 1.00 1.00 Moodle 4.40 1.00 1.00 1.00 SEVIMA EdLink 3.70 1.00 1.00 1.00 The purpose of organizing online learning Microsoft Teams 4.10 1.00 1.00 1.00 Kelase 3.70 1.00 1.00 1.00 Moodle 4.60 1.00 1.00 1.00 SEVIMA EdLink 3.50 1.00 1.00 1.00 Support from the academic community for the implementation of online learning Microsoft Teams 3.60 1.00 1.00 1.00 Kelase 3.40 1.00 1.00 1.00 Moodle 4.40 1.00 1.00 1.00 SEVIMA EdLink 3.20 1.00 1.00 1.00 The ability of the development teams to install and control the supporting devices for the realization of online learning Microsoft Teams 1.00 3.40 1.00 1.00 Kelase 1.00 2.90 1.00 1.00 Moodle 1.00 4.10 1.00 1.00 SEVIMA EdLink 1.00 2.80 1.00 1.00 Funding support from college Microsoft Teams 1.00 3.60 1.00 1.00 Kelase 1.00 3.20 1.00 1.00 Moodle 1.00 4.30 1.00 1.00 SEVIMA EdLink 1.00 3.30 1.00 1.00 Lecturer’s knowledge about online learning platforms Microsoft Teams 1.00 3.20 1.00 1.00 Kelase 1.00 2.60 1.00 1.00 Moodle 1.00 3.70 1.00 1.00 SEVIMA EdLink 1.00 2.70 1.00 1.00 https://www.microsoft.com/en/microsoft-teams/group-chat-software https://www.microsoft.com/en/microsoft-teams/group-chat-software https://moodle.org/ https://www.kelase.com/ https://edlink.id/ HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 240 Student’s knowledge about online learning platforms Microsoft Teams 1.00 3.30 1.00 1.00 Kelase 1.00 2.90 1.00 1.00 Moodle 1.00 4.20 1.00 1.00 SEVIMA EdLink 1.00 3.20 1.00 1.00 Lecturer skills in using online learning platforms Microsoft Teams 1.00 1.00 3.10 1.00 Kelase 1.00 1.00 2.50 1.00 Moodle 1.00 1.00 3.60 1.00 SEVIMA EdLink 1.00 1.00 2.60 1.00 Student skills in using online learning platforms Microsoft Teams 1.00 1.00 3.20 1.00 Kelase 1.00 1.00 2.70 1.00 Moodle 1.00 1.00 3.90 1.00 SEVIMA EdLink 1.00 1.00 2.90 1.00 The reporting mechanism for the use of supporting funds for the realization of online learning Microsoft Teams 1.00 1.00 2.80 1.00 Kelase 1.00 1.00 2.60 1.00 Moodle 1.00 1.00 3.70 1.00 SEVIMA EdLink 1.00 1.00 2.80 1.00 Lecturer satisfaction in using online learning platforms Microsoft Teams 1.00 1.00 1.00 3.10 Kelase 1.00 1.00 1.00 2.90 Moodle 1.00 1.00 1.00 3.60 SEVIMA EdLink 1.00 1.00 1.00 2.70 Student satisfaction in using online learning platforms Microsoft Teams 1.00 1.00 1.00 3.30 Kelase 1.00 1.00 1.00 3.10 Moodle 1.00 1.00 1.00 3.80 SEVIMA EdLink 1.00 1.00 1.00 2.90 Satisfaction of the development teams in managing the online learning platform Microsoft Teams 1.00 1.00 1.00 3.50 Kelase 1.00 1.00 1.00 3.70 Moodle 1.00 1.00 1.00 4.20 SEVIMA EdLink 1.00 1.00 1.00 3.30 Quality of online learning using online platforms Microsoft Teams 1.00 1.00 1.00 3.70 Kelase 1.00 1.00 1.00 3.90 Moodle 1.00 1.00 1.00 4.40 SEVIMA EdLink 1.00 1.00 1.00 3.40 Table 2 shows the evaluation aspects used to measure the quality of several online platforms in view of the CIPP evaluation component. There were four platforms whose quality was measured, including Microsoft Teams, Kelase, Moodle, and SEVIMA EdLink. The average importance rating score shown for each evaluation component in Table 2 was obtained from the assessment scores given by 20 respondents, consisting of 10 informatics experts and 10 education experts. 2) Determination of Weights from Experts that had been Revised for Each Evaluation Component Based on Equation 1, it can be determined the weight given by the experts that had been corrected/improved for each CIPP evaluation component. The results of the weights that had been corrected can be seen in Table 3. Table 3. Weights from Experts that had been Revised Evaluation Components Weight Value from Each Expert  Weights from Experts that had been Revised Expert- 1 Expert- 2 Expert- 3 Expert- 4 Expert-5 Expert- 6 Context 5 4 5 5 5 5 29 0.257 Input 4 5 5 5 4 4 27 0.239 Process 5 5 4 5 5 4 28 0.248 Product 5 5 5 4 5 5 29 0.257 Total 113 1 HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 241 Table 3 shows the weighted repair scores for each evaluation component. Giving a weight repair score was carried out by six experts. The weight repair score for the Context component is obtained by the following calculation:  Context component /  Total, so the weight repair score for the context component = 29/113 = 0.257. And so on, the same calculation is performed for Input, Process, and Product components. Weight repair score for the Input component = 27/113 = 0.239. Weight repair score for the Process component = 28/113 = 0.248. Weight repair score for the Product component = 29/113 = 0.257. The total weight repair for all CIPP evaluation components must be valuable of 1, to comply with the conditions set out in Equation 2, where wj must be valuable of 1. 3) Calculation of S Vector Referring to Equation 2, the data in Tables 2 and 3 can be calculated of normalization to get the S vector. The calculation of the S vector can be shown as follows. S1 = (3.900.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.418; S2 = (3.800.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.409 S3 = (4.400.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.463; S4 = (3.700.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.399 S5 = (4.100.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.436; S6 = (3.700.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.399 S7 = (4.600.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.479; S8 = (3.500.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.379 S9 = (3.600.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.389; S10 = (3.400.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.369 S11 = (4.400.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.463; S12 = (3.200.257) × (1.000.239) × (1.000.248) × (1.000.257) = 1.348 S13 = (1.000.257) × (3.400.239) × (1.000.248) × (1.000.257) = 1.340; S14 = (1.000.257) × (2.900.239) × (1.000.248) × (1.000.257) = 1.290 S15 = (1.000.257) × (4.100.239) × (1.000.248) × (1.000.257) = 1.401; S16 = (1.000.257) × (2.800.239) × (1.000.248) × (1.000.257) = 1.279 S17 = (1.000.257) × (3.600.239) × (1.000.248) × (1.000.257) = 1.358; S18 = (1.000.257) × (3.200.239) × (1.000.248) × (1.000.257) = 1.320 S19 = (1.000.257) × (4.300.239) × (1.000.248) × (1.000.257) = 1.417; S20 = (1.000.257) × (3.300.239) × (1.000.248) × (1.000.257) = 1.330 S21 = (1.000.257) × (3.200.239) × (1.000.248) × (1.000.257) = 1.320; S22 = (1.000.257) × (2.600.239) × (1.000.248) × (1.000.257) = 1.256 S23 = (1.000.257) × (3.700.239) × (1.000.248) × (1.000.257) = 1.367; S24 = (1.000.257) × (2.700.239) × (1.000.248) × (1.000.257) = 1.268 S25 = (1.000.257) × (3.300.239) × (1.000.248) × (1.000.257) = 1.330; S26 = (1.000.257) × (2.900.239) × (1.000.248) × (1.000.257) = 1.290 S27 = (1.000.257) × (4.200.239) × (1.000.248) × (1.000.257) = 1.409; S28 = (1.000.257) × (3.200.239) × (1.000.248) × (1.000.257) = 1.320 S29 = (1.000.257) × (1.000.239) × (3.100.248) × (1.000.257) = 1.324; S30 = (1.000.257) × (1.000.239) × (2.500.248) × (1.000.257) = 1.255 S31 = (1.000.257) × (1.000.239) × (3.600.248) × (1.000.257) = 1.374; S32 = (1.000.257) × (1.000.239) × (2.600.248) × (1.000.257) = 1.267 S33 = (1.000.257) × (1.000.239) × (3.200.248) × (1.000.257) = 1.334; S34 = (1.000.257) × (1.000.239) × (2.700.248) × (1.000.257) = 1.279 S35 = (1.000.257) × (1.000.239) × (3.900.248) × (1.000.257) = 1.401; S36 = (1.000.257) × (1.000.239) × (2.900.248) × (1.000.257) = 1.302 S37 = (1.000.257) × (1.000.239) × (2.800.248) × (1.000.257) = 1.291; S38 = (1.000.257) × (1.000.239) × (2.600.248) × (1.000.257) = 1.267 S39 = (1.000.257) × (1.000.239) × (3.700.248) × (1.000.257) = 1.383; S40 = (1.000.257) × (1.000.239) × (2.800.248) × (1.000.257) = 1.291 S41 = (1.000.257) × (1.000.239) × (1.000.248) × (3.100.257) = 1.337; S42 = (1.000.257) × (1.000.239) × (1.000.248) × (2.900.257) = 1.314 S43 = (1.000.257) × (1.000.239) × (1.000.248) × (3.600.257) = 1.389; S44 = (1.000.257) × (1.000.239) × (1.000.248) × (2.700.257) = 1.290 S45 = (1.000.257) × (1.000.239) × (1.000.248) × (3.300.257) = 1.359; S46 = (1.000.257) × (1.000.239) × (1.000.248) × (3.100.257) = 1.337 S47 = (1.000.257) × (1.000.239) × (1.000.248) × (3.800.257) = 1.409; S48 = (1.000.257) × (1.000.239) × (1.000.248) × (2.900.257) = 1.314 S49 = (1.000.257) × (1.000.239) × (1.000.248) × (3.500.257) = 1.379; S50 = (1.000.257) × (1.000.239) × (1.000.248) × (3.700.257) = 1.399 S51 = (1.000.257) × (1.000.239) × (1.000.248) × (4.200.257) = 1.445; S52 = (1.000.257) × (1.000.239) × (1.000.248) × (3.300.257) = 1.359 S53 = (1.000.257) × (1.000.239) × (1.000.248) × (3.700.257) = 1.399; S54 = (1.000.257) × (1.000.239) × (1.000.248) × (3.900.257) = 1.418 S55 = (1.000.257) × (1.000.239) × (1.000.248) × (4.400.257) = 1.463; S56 = (1.000.257) × (1.000.239) × (1.000.248) × (3.400.257) = 1.369 S = S1 + S2 + S3 + S4 + S5 + S6 + S7 + S8 + S9 + S10 + S11 + S12 + S13 + S14 + S15 + S16 + S17 + S18 + S19 + S20 + S21 + S22 + S23 + S24 + S25 + S26 + S27 + S28 + S29 + S30 + S31 + S32 + S33 + S34 + S35 + S36 + S37 + S38 + S39 + S40 + S41 + S42 + S43 + S44 + S45 + S46 + S47 + S48 + S49 + S50 + S51 + S52 + S53 + S54 + S55 + S56 = 75.993 HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 242 4) Calculation of V Vector Based on Equation 3 and the value of the S vector from each evaluation aspect, so can be determined the V vector. The calculation of the V vector can be shown as follows. V1 = S1 / S = 1.418/75.993 = 0.0187; V2 = S2 / S = 1.409/75.993 = 0.0185; V3 = S3 / S = 1.463/75.993 = 0.0192 V4 = S4 / S = 1.399/75.993 = 0.0184; V5 = S5 / S = 1.436/75.993 = 0.0189; V6 = S6 / S = 1.399/75.993 = 0.0184 V7 = S7 / S = 1.479/75.993 = 0.0195; V8 = S8 / S = 1.379/75.993 = 0.0181; V9 = S9 / S = 1.389/75.993 = 0.0183 V10 = S10 / S = 1.369/75.993 = 0.0180; V11 = S11 / S = 1.463/75.993 = 0.0192; V12 = S12 / S = 1.348/75.993 = 0.0177 V13 = S13 / S = 1.340/75.993 = 0.0176; V14 = S14 / S = 1.290/75.993 = 0.0170; V15 = S15 / S = 1.401/75.993 = 0.0184 V16 = S16 / S = 1.279/75.993 = 0.0168; V17 = S17 / S = 1.358/75.993 = 0.0179; V18 = S18 / S = 1.320/75.993 = 0.0174 V19 = S19 / S = 1.417/75.993 = 0.0186; V20 = S20 / S = 1.330/75.993 = 0.0175; V21 = S21 / S = 1.320/75.993 = 0.0174 V22 = S22 / S = 1.256/75.993 = 0.0165; V23 = S23 / S = 1.367/75.993 = 0.0180; V24 = S24 / S = 1.268/75.993 = 0.0167 V25 = S25 / S = 1.330/75.993 = 0.0175; V26 = S26 / S = 1.290/75.993 = 0.0170; V27 = S27 / S = 1.409/75.993 = 0.0185 V28 = S28 / S = 1.320/75.993 = 0.0174; V29 = S29 / S = 1.324/75.993 = 0.0174; V30 = S30 / S = 1.255/75.993 = 0.0165 V31 = S31 / S = 1.374/75.993 = 0.0181; V32 = S32 / S = 1.267/75.993 = 0.0167; V33 = S33 / S = 1.334/75.993 = 0.0176 V34 = S34 / S = 1.279/75.993 = 0.0168; V35 = S35 / S = 1.401/75.993 = 0.0184; V36 = S36 / S = 1.302/75.993 = 0.0171 V37 = S37 / S = 1.291/75.993 = 0.0170; V38 = S38 / S = 1.267/75.993 = 0.0167; V39 = S39 / S = 1.383/75.993 = 0.0182 V40 = S40 / S = 1.291/75.993 = 0.0170; V41 = S41 / S = 1.337/75.993 = 0.0176; V42 = S42 / S = 1.314/75.993 = 0.0173 V43 = S43 / S = 1.389/75.993 = 0.0183; V44 = S44 / S = 1.290/75.993 = 0.0170; V45 = S45 / S = 1.359/75.993 = 0.0179 V46 = S46 / S= 1.337/75.993 = 0.0176; V47 = S47 / S = 1.409/75.993 = 0.0185; V48 = S48 / S = 1.314/75.993 = 0.0173 V49 = S49 / S= 1.379/75.993 = 0.0181; V50 = S50 / S = 1.399/75.993 = 0.0184; V51 = S51 / S = 1.445/75.993 = 0.0190 V52 = S52 / S = 1.359/75.993 = 0.0179; V53 = S53 / S = 1.399/75.993 = 0.0184; V54 = S54 / S = 1.418/75.993 = 0.0187 V55 = S55 / S = 1.463/75.993 = 0.0192; V56 = S56 / S = 1.369/75.993 = 0.0180 5) Determination of the Best Platform Based on the value of the V vector in each evaluation aspect, so can be carried out the process of determining the best online platform. The best platform is determined based on the highest score of the V vector. Recapitulation of the V vector for each online platform based on evaluation aspects can be seen in Table 4. Table 4. Recapitulation of the V Vector for each Online Platform Based on Evaluation Aspects Evaluation Aspects Platforms V vector Vision and mission of organizing online learning Microsoft Teams 0.0187 Kelase 0.0185 Moodle 0.0192 SEVIMA EdLink 0.0184 The purpose of organizing online learning Microsoft Teams 0.0189 Kelase 0.0184 Moodle 0.0195 SEVIMA EdLink 0.0181 Support from the academic community for the implementation of online learning Microsoft Teams 0.0183 Kelase 0.0180 Moodle 0.0192 SEVIMA EdLink 0.0177 The ability of the development teams to install and control the supporting devices for the realization of online learning Microsoft Teams 0.0176 Kelase 0.0170 Moodle 0.0184 SEVIMA EdLink 0.0168 HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 243 Funding support from college Microsoft Teams 0.0179 Kelase 0.0174 Moodle 0.0186 SEVIMA EdLink 0.0175 Lecturer’s knowledge about online learning platforms Microsoft Teams 0.0174 Kelase 0.0165 Moodle 0.0180 SEVIMA EdLink 0.0167 Student’s knowledge about online learning platforms Microsoft Teams 0.0175 Kelase 0.0170 Moodle 0.0185 SEVIMA EdLink 0.0174 Lecturer skills in using online learning platforms Microsoft Teams 0.0174 Kelase 0.0165 Moodle 0.0181 SEVIMA EdLink 0.0167 Student skills in using online learning platforms Microsoft Teams 0.0176 Kelase 0.0168 Moodle 0.0184 SEVIMA EdLink 0.0171 The reporting mechanism for the use of supporting funds for the realization of online learning Microsoft Teams 0.0170 Kelase 0.0167 Moodle 0.0182 SEVIMA EdLink 0.0170 Lecturer satisfaction in using online learning platforms Microsoft Teams 0.0176 Kelase 0.0173 Moodle 0.0183 SEVIMA EdLink 0.0170 Student satisfaction in using online learning platforms Microsoft Teams 0.0179 Kelase 0.0176 Moodle 0.0185 SEVIMA EdLink 0.0173 Satisfaction of the development teams in managing the online learning platform Microsoft Teams 0.0181 Kelase 0.0184 Moodle 0.0190 SEVIMA EdLink 0.0179 Quality of online learning using online platforms Microsoft Teams 0.0184 Kelase 0.0187 Moodle 0.0192 SEVIMA EdLink 0.0180 The highest score of the V vector shown in Table 4 was 0.0195. This clearly showed that the best online platform that was able to be used to support online learning during the COVID-19 pandemic was Moodle (shown by green block in Table 4). The score of 0.0195 was obtained from the evaluation aspect of the “purpose of implementing online learning”. This indicates that the Moodle platform is very appropriate to use supporting the realization of the goals of organizing online learning. 3.3. Quality Assessment of the Weighted Product Method Simulation Calculation The quality of the Weighted Product calculation simulation was assessed by 20 experts. The tool used by the expert to assess was a questionnaire consisting of eight questions. The quality assessment results of the Weighted Product method simulation calculation can be seen in Table 5. HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 244 Table 5. The Quality Assessment Results of the Weighted Product Method Simulation Calculation No Respondents Items-  Percentage of Quality (%) I1 I2 I3 I4 I5 I6 I7 I8 1 Educational Expert-1 5 5 4 5 4 4 4 5 36 90.000 2 Educational Expert-2 4 5 4 4 5 5 4 4 35 87.500 3 Educational Expert-3 5 5 5 4 4 4 5 4 36 90.000 4 Educational Expert-4 4 4 4 4 4 5 4 4 33 82.500 5 Educational Expert-5 4 4 5 5 4 5 4 5 36 90.000 6 Educational Expert-6 4 4 5 4 5 4 5 4 35 87.500 7 Educational Expert-7 4 4 4 5 5 5 4 4 35 87500 8 Educational Expert-8 4 5 4 4 4 4 4 4 33 82.500 9 Educational Expert-9 4 5 5 4 4 4 4 4 34 85.000 10 Educational Expert-10 4 4 4 5 4 4 4 4 33 82.500 11 Informatics Expert-1 4 4 5 5 4 4 5 4 35 87.500 12 Informatics Expert-2 4 4 5 4 5 5 4 4 35 87.500 13 Informatics Expert-3 5 5 4 4 4 4 4 4 34 85.000 14 Informatics Expert-4 5 5 5 4 4 4 4 4 35 87.500 15 Informatics Expert-5 5 5 5 5 4 5 4 4 37 92.500 16 Informatics Expert-6 4 4 5 4 5 4 5 4 35 87.500 17 Informatics Expert-7 5 4 4 4 5 4 4 5 35 87.500 18 Informatics Expert-8 5 4 4 4 5 4 4 4 34 85.000 19 Informatics Expert-9 5 4 4 4 5 4 5 4 35 87.500 20 Informatics Expert-10 4 5 5 4 5 5 4 5 37 92.500 Average 87.250 Based on the average percentage of quality shown in Table 5, it was able to be stated that the quality of the Weighted Product calculation simulation was categorized as very good when viewed from the quality standard refers to eleven’s scale. In addition, when viewed from the simulation results of the Weighted Product calculation, it was found that the best online platform that was able to be used to support the online learning process during the COVID-19 pandemic was Moodle. If the results of this research are compared with Vydia et al.’s [45] research, there are certainly similarities and differences. The similarity between this research and Vydia et al.’s research is that both use decision-support methods in choosing an online platform. The difference is that this study combines the educational evaluation model “CIPP” with a decision support method “Weighted Product” in determining the best online platform to support the learning process. Meanwhile, research by Vydia et al. only uses decision support methods (F-MADM/Fuzzy Multiple Attribute Decision Making) in determining online platforms to support the learning process. In principle, this research has similarities with the research of Ouadoud et al. [46], which shows several online platforms that can be used to support the learning process. However, the difference is that Ouadoud et al.’s research does not show in detail the best online platform that can be used to support online learning. Meanwhile, this research has shown that there is a best online platform; there is even complete evidence of a calculation process to get the best online platform. Satria’s [47] research shows the best online platform can be used for learning in the new normal era. In principle, Satria’s research and the results of this study have similarities in determining the best platform. However, the difference is the mechanism or method used to get the best online platform. The results of this research have an advantage when compared to Satria’s research results, namely in the calculation process used to make decisions about the best online platform. This research uses a combination of educational evaluation models and decision support methods to obtain accurate calculation results in determining the best online platform. Meanwhile, Satria’s research only used respondents’ perception scores, which were obtained using an instrument in the form of questionnaires. The results of this research were strengthened by several other studies, such as the research of Kurniawan & Septiana [25], Ardinengtyas & Himawan [48], Sirwan et al. [49], Simanjuntak & Perwira [50], Quansah & Essiam [51], Amin et al. [52], Putri et al. [53], Makruf et al. [54], Dascalu et al. [55], and Mpungose [56], which principle stated that Moodle was an online platform that was suitable for use during the COVID-19 pandemic to support online learning. HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 245 Based on the several advantages of the results of this research and the existence of strengthening support from several previous studies, the novelty of this research is the existence of an educational evaluation model that is combined with one of the methods in a decision support system called the Weighted Product. This model can be used to determine the best online platform to support the online learning process in the education field generally and in health colleges specifically. The limitation of this research is that it is difficult to determine the best online platform if there are V vectors that have the same value. 4. Conclusion Generally, the results of this research showed a very good simulation of the Weighted Product method calculation. The results of this categorization show the positive significance of this results study which are useful for convincing the public regarding the best online platforms that can be used to support the learning process in health colleges. This positive significance is confirmed by the result of a quality percentage of 87.250% in the range of 85–94% when referring to the eleven-scale quality standard. Theoretically, the results of this research make a positive contribution to science and technology by demonstrating a combination of knowledge between educational evaluation models combined with decision support system methods. The combination of two pieces of knowledge produces an accurate calculation process for determining the best online platform that is useful in supporting a better learning process for the advancement of education. Practically, future work can be done by researchers, the academic community, or educational observers to overcome the obstacle of this research, which is to determine the best online platform based on platform priority if the V vector values are the same. In addition to referring to the V vector value, it is better if the evaluation aspect that is a priority to support the success of the online learning implementation also needs to be used as a determinant of the best online platform selection. The novelty of this research is the combination of the Weighted Product method and the CIPP model, which can produce accurate recommendations to make it easier for educational evaluators to determine the best online platform that supports online learning during the COVID-19 pandemic and after the COVID-19 pandemic. The impact of these research results on the field of education is new knowledge for education evaluators to use the product weighted method combined with the educational evaluation model in conducting an evaluation. 5. Declarations 5.1. Author Contributions Conceptualization, D.G.H.D. and P.W.A.S.; methodology, D.G.H.D.; formal analysis, D.G.H.D., P.W.A.S., N.S., M.S.L.A., I.M.S., I.N.D.A., N.W.R.M., and G.A.D.S.; investigation, D.G.H.D.; data curation, D.G.H.D., P.W.A.S., N.S., M.S.L.A., I.M.S., I.N.D.A., N.W.R.M., and G.A.D.S.; writing—original draft preparation, D.G.H.D.; writing—review and editing, D.G.H.D. All authors have read and agreed to the published version of the manuscript. 5.2. Data Availability Statement The data presented in this study are available in the article. 5.3. Funding The authors received no financial support for the research, authorship, and/or publication of this article. 5.4. Acknowledgements The authors express their gratitude to the Rector and Chair of the Research and Community Service Institute in each college, including Universitas Pendidikan Ganesha, Sekolah Tinggi Ilmu Kesehatan Buleleng, Universitas Pendidikan Nasional, and Institut Pariwisata dan Bisnis Internasional who give permission and the opportunity for the authors to complete this collaborative research. 5.5. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. 6. References [1] Sundari, H. D., & Utomo, P. (2020). Five E-Learning for Education in Indonesia. Advances in Social Science, Education and Humanities Research, 440, 48–52. doi:10.2991/assehr.k.200521.010. [2] Divayana, D. G. H., Adiarta, A., & Suyasa, P. W. A. (2021). Development of Material Contents and Online Assessment Based on the Sevima Edlink Platform for Online Learning of Program Evaluation Subject During COVID-19 Pandemic in Indonesia. Journal of Technology and Science Education, 11(2), 498–512. doi:10.3926/jotse.1243. HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 246 [3] Jannah, S. N., Sobandi, A., & Suwatno, S. (2020). The Measurement of Usability Using USE Questionnaire on the Google Classroom Application as E-learning Media (A Case study: SMK Negeri 1 Bandung). Teknodika, 18(2), 94-105. doi:10.20961/teknodika.v18i2.42486. [4] Sukawatie, L. (2018). Using Quipper School Website as an Online Assessment for English Teaching and Learning. The 2nd English Language and Literature International Conference, Atlantis Press, Amsterdam, Netherlands, 263–270. [5] Purwaningsih, H., & Dardjito, H. (2021). Implementation of CIPP Model for Online Learning Evaluation During COVID-19 Pandemic. Getsempena English Education Journal, 8(2), 294–309. doi:10.46244/geej.v8i2.1394. [6] Damayanti, E., Ibrahim, M. M., & Ismail, M. I. (2022). Evaluation of Online Learning Programs at Universities Using the CIPP Model. Jurnal Educative: Journal of Educational Studies, 6(1), 95-110. doi:10.30983/educative.v6i1.4678. [7] Anh, V. T. K., & Pang, V. (2021). The Application of CIPP Model to Evaluate Online Teaching for English Majored Programs in Vietnam During the COVID-19 Pandemic. Journal of Institutional Research South East Asia, 19(2), 146-165. [8] Prayogo, D., Rusdarti, Raharjo, T. J., & Kuswardinah, A. (2022). Distance Learning Evaluation with The CIPP Model. Advances in Social Science, Education and Humanities Research, 574, 330-336. doi:10.2991/assehr.k.211125.062. [9] DeCoito, I., & Estaiteyeh, M. (2022). Online Teaching During the COVID-19 Pandemic: Exploring Science/STEM Teachers’ Curriculum and Assessment Practices in Canada. Disciplinary and Interdisciplinary Science Education Research, 4(1), 1–18. doi:10.1186/s43031-022-00048-z. [10] Toan, P. N., Dang, T. T., & Hong, L. T. T. (2021). E-learning Platform Assessment and Selection Using Two-Stage Multi- Criteria Decision-Making Approach with Grey Theory: A Case Study in Vietnam. Mathematics, 9(23), 1–20. doi:10.3390/math9233136. [11] Ong, A.K.S., Prasetyo, Y.T., Chuenyindee, T., Young, M.N., Doma, B.T., Caballes, D.G., Centeno, R. S., Morfe, A.S., & Bautista, C.S. (2022). Preference Analysis on the Online Learning Attributes Among Senior High School Students During the COVID-19 Pandemic: A Conjoint Analysis Approach. Evaluation and Program Planning, 92, 102100, 1–9. doi:10.1016/j.evalprogplan.2022.102100. [12] Nguyen, L. D., & Nguyen, L. V. (2022). Schoology as an Online Learning Platform to Enhance English Language Ability for Undergraduates in Vietnam. Computer Assisted Language Learning Electronic Journal, 23(4), 139-161. [13] Shashiprabha, T. A. S., Rupasinghe, L., Liyanage, I. M., & Liyanapathirana, C. (2020). Student Feedback Analyzer for E- Learning Platforms. International Journal of Engineering Research & Technology, 9(12), 472–477. [14] Cabual, R. A., & Cabual, M. M. A. (2022). The Extent of the Challenges in Online Learning during the COVID-19 Pandemic. OALib, 09(01), 1–13. doi:10.4236/oalib.1108233. [15] Herdiansah, A., Handayani, N., & Kurniawan, A. (2019). A Development of Decision Support Systems Selection of Employee Acceptance Using Weighted Product Method. Journal of Information Systems and Informatics, 1(2), 87–97. doi:10.33557/journalisi.v1i2.10. [16] Saputra, D., Kudiantoro Widianto, Tyas Setiyorini, & Ibnu Alfarobi. (2021). Decision Support System for Cloud Computing Service Selection Using the Weighted Product Method (Case Study: PT. Deptech Digital Indonesia). International Journal of Science, Technology & Management, 2(1), 78–92. doi:10.46729/ijstm.v2i1.103. [17] Sembiring, A. P., Tulus, Sembiring, R. W., & Maulana, H. (2018). Rule Model with Fuzzy Simple Additive Weighting Approach and Weighted Product on Determination. International Journal of Latest Trends in Engineering and Technology, 10(1), 54–62. doi:10.21172/1.101.10. [18] Sisodia, G. (2018). Intuitionistic Fuzzy Weighted Sum and Product Method for Electronic Service Quality Selection Problem. International Journal of Modern Education and Computer Science, 10(9), 33–43. doi:10.5815/ijmecs.2018.09.05. [19] Anifah, L., & Haryanto. (2020). Decision Support System Cattle Weight Prediction using Artificial Selected Weighting Method. 2020 Third International Conference on Vocational Education and Electrical Engineering (ICVEE), Surabaya, Indonesia, 1-6. doi:10.1109/icvee50212.2020.9243263. [20] Murdani, ., Sianturi, F. A., Hutahaean, H. D., Sinaga, S. B., & Rajagukguk, D. M. (2018). Implementation of the Weighted Product Method in the Best Student Selection Decision Making System Application. Proceedings of the 1st Unimed International Conference on Economics Education and Social Science, 99-104. doi:10.5220/0009495900990104. [21] Sa’ad, M. I., Bryan, D., Kusrini, & Supriatin. (2020). Decision Support System for Covid19 Affected Family Cash Aid Recipients Using the Naïve Bayes Algorithm and the Weight Product Method. 2020 3rd International Conference on Information and Communications Technology (ICOIACT), 120-125. doi:10.1109/icoiact50329.2020.9331964. [22] Verma, S., & Patel, K. (2019). Weighted product Taxonomy for Mobile-Commerce site in Recommendation of Product based on Heuristic Approach. 2019 International Conference on Intelligent Computing and Control Systems (ICCS), 1435-1440. doi:10.1109/iccs45141.2019.9065437. HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 247 [23] Zhao, Z., Zhang, H., & Yu, Y. (2020). Method for Calculating Text Similarity Of Cross-Weighted Products Applied To Power Grid Model Search. 2020 IEEE 4th Conference on Energy Internet and Energy System Integration (EI2), Wuhan, China, 3863- 3867. doi:10.1109/ei250167.2020.9346920. [24] Fitriasari, N. S., Fitriani, S. A., & Sukamto, R. A. (2017). Comparison of Weighted Product Method and Technique for Order Preference by Similarity to Ideal Solution Method: Complexity and Accuracy. 2017 3rd International Conference on Science in Information Technology (ICSITech), 453–458. doi:10.1109/icsitech.2017.8257155. [25] Kurniawan, B. I., & Septiana, A. R. (2021). An Analysis of Students’ Experience in the Use of Moodle in Writing Class during Pandemic COVID-19. Linguists : Journal Of Linguistics and Language Teaching, 7(2), 16-28. doi:10.29300/ling.v7i2.5318. [26] Malik, Z., Najmi, E., Hashmi, K., Rafique, M. M., & Rezgui, A. (2022). Product Weighted Taxonomy Extraction using Twitter. International Journal of Business Information Systems, 39(4), 497–515. doi:10.1504/IJBIS.2022.122875. [27] Maulana, H. O., Shofi, I. M., Rozy, N. F., & Agustin, F. E. M. (2017). Application for determining mustahiq based on the priority using weight product method (case study: BAZIS DKI Jakarta). 2017 5th International Conference on Cyber and IT Service Management (CITSM), 1-4. doi:10.1109/citsm.2017.8089315. [28] Nababan, L., & Tuti, E. (2018). Determination Feasibility of Poor Household Surgery By Using Weighted Product Method. 2018 6th International Conference on Cyber and IT Service Management (CITSM), 1-6. doi:10.1109/citsm.2018.8674253. [29] Ramadhan, I., & Zaky, U. (2020). Method Comparison Analysis Simple Additive Weighting (SAW) With Weighted Product (WP) Method in Supporting the Decision To Accept New Employees. International Journal of Engineering Technology and Natural Sciences, 2(1), 11–17. doi:10.46923/ijets.v2i1.66. [30] Solikhun, S. (2017). Decision support system in Predicting the Best teacher with Multi Atribute Decesion Making Weighted Product (MADMWP) Method. International Journal of Artificial Intelligence Research, 1(1), 6-10. doi:10.29099/ijair.v1i1.1. [31] Utomo, P., & Budiman, A. (2020). Application of Weighted Product (WP) Method in Decision Support of Unmer Madiun Rector Election. Research : Journal of Computer, Information System & Technology Management, 3(2), 74-82. doi:10.25273/research.v3i2.6700. [32] Kurniawan, R., Windarto, A. P., Fauzan, M., Solikhun, S., & Damanik, I. S. (2019). Analysis of Weight Product (WP) Algorithms in the best Go Car Driver Recommendations at PT. Maranatha Putri Bersaudara. International Journal of Information System & Technology, 3(1), 18-24. doi:10.30645/ijistech.v3i1.28. [33] Dalimunte, M., & Salmiah, M. (2019). Students’ Ability at Changing Direct into Indirect Speech and Indirect into Direct Speech. Budapest International Research and Critics Institute (BIRCI-Journal) : Humanities and Social Sciences, 2(2), 178–185. doi:10.33258/birci.v2i2.249. [34] Divayana, D. G. H., Adiarta, A., & Suyasa, P. W. A. (2022). Development of Discrepancy Evaluation Model Based on Tat Twam Asi with TOPSIS Calculation. International Journal of Advanced Computer Science and Applications, 13(7), 737–752. doi:10.14569/IJACSA.2022.0130787. [35] Sari, S. A., & Rezeki, Y. S. (2019). The Development of an Ingenious Circuit Based on Chemo-Edutainment Learning. International Journal of Educational Research Review, 4(1), 15–25. doi:10.24331/ijere.467078. [36] Sugiharni, G. A. D. (2018). The Development of Interactive Instructional Media Oriented to Creative Problem Solving Model on Function Graphic Subject. Journal of Education Research and Evaluation, 2(4), 183–189. doi:10.23887/jere.v2i4.16694. [37] Sujariati, S. (2020). Improving the Students’ Listening Comprehension through Aural-Oral Language Approach. Linguistic, English Education and Art (LEEA) Journal, 3(2), 486–505. doi:10.31539/leea.v3i2.1253. [38] Sutirna, S. (2019). Subject Teachers’ Perceptions of Academic Mentoring and Counseling Services. COUNS-EDU: The International Journal of Counseling and Education, 4(4), 129–133. doi:10.23916/0020190423040. [39] Yulina, I. K., Permanasari, A., Hernani, H., & Setiawan, W. (2019). Analytical Thinking Skill Profile and Perception of Pre Service Chemistry Teachers in Analytical Chemistry Learning. Journal of Physics: Conference Series, 1157, 042046, 1–7. doi:10.1088/1742-6596/1157/4/042046. [40] Divayana, D. G. H., Ariawan, I. P. W., & Giri, M. K. W. (2021). CIPP-SAW Application as an Evaluation Tool of E-learning Effectiveness. International Journal of Modern Education and Computer Science, 13(6), 42–59. doi:10.5815/ijmecs.2021.06.05. [41] Mantasiah, R., Yusri, Y., & Jufri, J. (2020). Semantic Feature Analysis Model: Linguistics Approach in Foreign Language Learning Material Developmen. International Journal of Instruction, 13(1), 185–196. doi:10.29333/iji.2020.13112a. [42] Nawawi, S., Nizkon, & Azhari, A. T. (2020). Analysis of the Level of Critical Thinking Skills of Students in Biological Materials at Muhammadiyah High School in Palembang City. Universal Journal of Educational Research, 8(3D), 47–53. doi:10.13189/ujer.2020.081707. HighTech and Innovation Journal Vol. 4, No. 1, March, 2023 248 [43] Sari, L. Y., Susanti, D., Fitriani, V., Supriatno, B., & Riandi, R. (2020). How to Validity Handbook in Introduction and Laboratory Techniques Oriented PBL. International Journal of Progressive Sciences and Technologies, 19(1), 250–254. [44] Divayana, D. G. H., Sudirtha, I. G., & Gading, I. K. (2020). Application Design of Countenance Evaluation Based on Tri Hita Karana-Aneka for Evaluating the Students’ Computer Capability and Students’ Character. Cogent Psychology, 7(1), 1–18. doi:10.1080/23311908.2020.1773095. [45] Vydia, V., Susanto, S., Handayani, S., & Alam, M. B. (2022). The Selection of Learning Platforms to Support Learning Using Fuzzy Multiple Attribute Decision Making. International Journal of Quantitative Research and Modeling, 3(1), 45–50. doi:10.46336/ijqrm.v3i1.257. [46] Ouadoud, M., Rida, N., & Chafiq, T. (2021). Overview of E-learning Platforms for Teaching and Learning. International Journal of Recent Contributions from Engineering, Science & IT (IJES), 9(1), 50-70. doi:10.3991/ijes.v9i1.21111. [47] Satria, B. (2021). Best Digital Platforms in ELT during New Norms Era for Remote Area. Journal of Language, Literature, and English Teaching (JULIET), 2(1), 1–5. doi:10.31629/jjumrah.v2i1.3143. [48] Ardinengtyas, A., & Himawan, A. N. (2021). Enhancing ELT Classroom Using Moodle E-Learning During the Pandemic: Students’ and Teachers’ Voices. IJEE (Indonesian Journal of English Education), 1(1), 24–39. doi:10.15408/ijee.v1i1.20220. [49] Sirwan, S., Ahyani, R., & Sartika, S. (2021). Development of Virtual Learning System (VLS) Based on Moodle as a Platform Online Learning in the Covic-19. Akademika, 10(02), 447–463. doi:10.34005/akademika.v10i02.1579. [50] Simanjuntak, O. S., & Perwira, R. I. (2020). Development of Spada Wimaya Online Learning Course Based on Moodle During and After the COVID-19 Pandemic. Proceeding of LPPM UPN “Veteran” Yogyakarta Conference Series 2020–Engineering and Science Series, 27 October, 2020, Yogyakarta, Indonesia, 677-683. [51] Quansah, R., & Essiam, C. (2021). The Use of Learning Management System (LMS) Moodle in the Midst of COVID-19 Pandemic: Students’ Perspective. Journal of Educational Technology and Online Learning, 4(3), 418–431. doi:10.31681/jetol.934730. [52] Amin, M., Sibuea, A. M., & Mustaqim, B. (2022). The Effectiveness of Online Learning Using E-Learning During Pandemic COVID-19. Journal of Education Technology, 6(2), 247–257. doi:10.23887/jet.v6i2.44125. [53] Putri, S. E., Hamuddin, B., Nursafira, M. S., & Derin, T. (2020). Discourse Analysis in E-Learning-Based Course Using Moodle Platform: An Experimental Design. REiLA : Journal of Research and Innovation in Language, 2(1), 19–26. doi:10.31849/reila.v2i1.3960. [54] Makruf, I., Rifa’i, A. A., & Triana, Y. (2022). Moodle-based Online Learning Management in Higher Education. International Journal of Instruction, 15(1), 135–152. doi:10.29333/iji.2022.1518a. [55] Dascalu, M.-D., Ruseti, S., Dascalu, M., McNamara, D. S., Carabas, M., Rebedea, T., & Trausan-Matu, S. (2021). Before and during COVID-19: A Cohesion Network Analysis of Students’ Online Participation in Moodle Courses. Computers in Human Behavior, 121, 106780, 1–19. doi:10.1016/j.chb.2021.106780. [56] Mpungose, C. B. (2020). Emergent Transition from Face-to-Face to Online Learning in a South African University in the Context of the Coronavirus Pandemic. Humanities and Social Sciences Communications, 7(1), 1–9. doi:10.1057/s41599-020-00603-x.