English Student Perception about the Effectiveness of Online Tutor Learning in Distance Education ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 11, Issue 1, February 2023 https://journal.uniku.ac.id/index.php/ERJEE 1 ENGLISH GRADUATE STUDENT’S PERCEPTION ABOUT THE EFFECTIVENESS OF ONLINE LEARNING TUTOR IN DISTANCE EDUCATION Fauzy Rahman Kosasih English Education Department of Universitas Terbuka, Tangerang Selatan, Indonesia Email: fauzyrahman@ecampus.ut.ac.id Juhana English Education Department of Universitas Terbuka, Tangerang Selatan, Indonesia Email: juhana@ecampus.ut.ac.id Zulmi Ramdani Department of Psychology, Faculty of Psychology, UIN Sunan Gunung Djati Bandung, Bandung, Indonesia Email: zulmiramdani@uinsgd.ac.id Lidwina Felisima Tae Faculty of Educational Sciences, Universitas of Timor, Nusa Tenggara Timur, Indonesia Email: lidwinafelisima@unimor.ac.id APA Citation: Kosasih, F. R., Juhana., Ramdani, Z., & Tae, L. F. (2023). English graduate student’s perception about the effectiveness of online learning tutor in distance education. English Review: Journal of English Education, 11(1), 1-8. https://doi.org/10.25134/erjee.v11i1.5531 Received: 28-08-2022 Accepted: 23-12-2022 Published: 28-02-2023 INTRODUCTION Distance learning is a lively issue and gets a lot of focus in carrying out the learning process, especially in a situation that full of uncertainty like what we experience today. The existence of distance learning has become the right choice, especially if the individuals and organizations that will organize the learning have limited time and certain distance (Mishra et al., 2020). As done by many schools, universities and institutions in today's era, distance learning can become the best choice to access the education quickly and easily (Budiastra et al., 2020; Churiyah et al., 2020; Lee et al., 2019). The distance learning model has been introduced for a long time, one of which is carried out by Universitas Terbuka. As a campus that uses a distance learning scheme, Universitas Terbuka has a curriculum structure, learning methods, learning media, and the interaction process that are carried out through online platforms (Charina et al., 2022; Prasojo et al., 2019; Rosyadi, 2018). Among the highlights of the campus is the term Tutor who is identified as an educator and the person who will Abstract: One of the successes of a learning program is obtained by the various feedbacks during the evaluation process, including when the program is implemented in the form of distance learning. The purpose of this study was to examine the perception of graduate students in English Education department about the effectiveness of distance learning carried out by the tutors during the learning process. Descriptive quantitative method was used in this study to answer the objectives of this study. A total of 21 students of master's degree in English Education department were involved in this study by selecting the respondents using the purposive sampling technique. The researchers used the measurement with reference to Kirkpatrick's model which explains the effectiveness of online learning through the aspects of reaction, learning, behavior and results. The result has shown that the majority of respondents were satisfied with the learning process provided by the tutor as seen from the satisfaction score that reached more than 75% for each aspect. Meanwhile, other results also showed that the four aspects which were the manifestation of the effectiveness of a tutor's learning have a strong and significant relationship in supporting the performance of a tutor. This study emphasized the importance of elaboration of every aspect that must be possessed by the tutors in carrying out online learning. Keywords: distance education; English graduate student; online learning tutor; student’s perception. Fauzy Rahman Kosasih, Juhana, Zulmi Ramdani, & Lidwina Felisima Tae English graduate student’s perception about the effectiveness of online learning tutor in distance education 2 provide the teaching process in a distance mode. This is what distinguishes it from learning in general which can be done face-to-face (Foo et al., 2021). Although this distance learning scheme has developed rapidly and many educational institutions are also using it, the limited information regarding the optimization and success of this distance learning program is one of the problems that still occur (Alyoussef, 2023; Lytras et al., 2022). In fact, if distance learning is effectively carried out, it will make it easier and reduce the cost of the implementation when it is done face-to-face. The success of a distance learning program will depend on many things. It starts from the purpose of planning a program, the availability of human resources in it, the connectivity between the program and the activities provided, and the existence of a structured and consistent program evaluation (Abuhammad, 2020; Suárez et al., 2021). The existence of an evaluation in a distance learning program is essential to obtain the information about whether or not a program is being held, including a specific evaluation of the human resources who run the program. Evaluation is basically a structured form to obtain the information about the sustainability of a program and to conclude that a program is good or not for the institutions and individuals in it (Dijk & Schodl, 2015; Martinez et al., 2016). Evaluation is an imperative part in a distance learning activity, because with this evaluation, the organization is able to predict things that are considered to be improved to get better productivity or goals. Various evaluations can be manifested in various activities and methods, so that the information obtained about the object of evaluation will be more comprehensive (Ramdani et al., 2019). Nowadays, many evaluations of distance learning programs are considered ineffective and assumed to be just a formality, even though essentially this evaluation activity can be very useful to get better results. This happens because there is an assumption that evaluation only costs a lot of money and the results are sometimes inaccurate. The inaccuracies in choosing the method, target object, and timing of the evaluation are considered to be a large part that makes the evaluation not optimal (Luo et al., 2021; Saidi et al., 2021; Vinogradova & Kliukas, 2015). In addition, it is the existence of the evaluation subject who is prone to subjective assessment which also adds to a bad impression in evaluating a program (Saidi et al., 2021; Vinogradova & Kliukas, 2015). One of the schemes that may be carried out by the institutions in evaluating the distance learning, is usually carried out to evaluate an educator or tutor. Tutor himself as presented in the previous explanation is a term that is commonly used to define an educator in an online mode (Fandiño & Velandia, 2020). Basically, they have the educational background and experience that are qualified to provide the teaching process. What distinguishes it may be from the status and formal identity that they do not have as in universities in general. The situation that occurs in a tutor then makes them more flexible to carry out the learning process according to what they want, even though there are rules that regulate these activities (Rosyadi, 2018). This is also one of the causes of the non-optimal evaluation carried out by the tutor (Fandiño & Velandia, 2020; Gould et al., 2014; Kang et al., 2021). The evaluation process carried out for the tutors usually only focuses on student perceptions in general, but it is still rare to use an assessment model with a focus on important psychological aspects in the distance learning process itself. In fact, when referring to the learning evaluation model introduced by the Kirkpatrick Model (Farjad, 2012; Kirkpatrick & Kayser-Kirkpatrick, 2014; Sakthi & Moshi, 2021), it is clear that to get optimization of the distance learning process includes various aspects including (1) reaction, which is a response that shows the level of individual satisfaction with the tutor so that they can feel the benefits of the tutor's presence; (2) learning, which is a form of individual assessment of the learning process in increasing the capacity of existing knowledge and skills; (3) behavior, is a form of real behavior that can be learned by individuals in the learning process and get comprehensive feedback, and (4) result, namely the level of achievement of the learning process that is generated and supports all existing processes. Many previous research has focused on the development of model evaluation in the distance learning process. Several similar studies have explained the major obstacles and challenges that will be faced in the distance learning (Harrison, 2020; Johnson & Cooke, 2016; Martin & Bolliger, 2018). In Indonesia itself, the online learning evaluation model has been discussed by many researchers and the majority contained the qualitative descriptions of how online learning was carried out (Abdullah, 2018; Affandi et al., 2020; Riyanto & Yunani, 2020). Meanwhile, the ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 11, Issue 1, February 2023 https://journal.uniku.ac.id/index.php/ERJEE 3 evaluation of online tutorials has also been carried out by Ardiasih et al. (2018), but focused on the approach model used based on the student needs. However, research that focuses on direct evaluation of educators is not widely carried out, especially by using a comprehensive approach as described by Kirkpatrick's model. The purpose of this study was to examine the perception of graduate students in English Education department regarding the effectiveness of a tutor in providing the distance learning. METHOD This study used a quantitative approach with a focus on descriptive quantitative design. This design was used to explain the descriptive data and then used it as supporting information for predetermined goals (Ramdani et al., 2021; Warsihna et al., 2021). This quantitative design is very useful for providing the initial information related to a topic, especially if the researcher does not have a lot of data and resources to explain a topic. But the researcher can objectively use a representative description to explain the initial situation of the phenomenon being raised. The subjects involved in this study were master students of English Education department in Universitas Terbuka who were attending the distance lectures and were active as the students. The samples were selected using a purposive sampling technique, so that the respondents who filled in were those who fit the predetermined characteristics (Kanya et al., 2021; Ramdani et al., 2021). The selected sample will then be given an informed consent sheet containing their willingness to participate in this research activity consciously and without any coercion. Meanwhile, the informed consent form itself has been validated by the Research and Community Service Institute of Universitas Terbuka. To answer the purpose of this study, the researcher used a research instrument in the form of a psychological questionnaire which was made using appropriate psychometric guidelines. The instrument made was based on a theoretical model of distance learning evaluation which was intended to see the effectiveness of a tutor in carrying out the learning process. The questionnaire was created using the theoretical foundation by Kirkpatrick Model which contains four aspects of distance learning (Kirkpatrick & Kayser-Kirkpatrick, 2014). The four aspects are reaction, learning, behavior and results. The questionnaire was made using a Likert scaling model consisting of 4 answer choices for each statement provided. The answer choices range from Strongly Disagree (1), Disagree (2), Agree (3), and Strongly Agree (4). Each assessment aspect has a different number of items, including the reaction aspect as many as 12 items, the learning aspect as many as 3 items, the behavioral aspect as many as 6 items, and the results aspect as many as 4 items. Thus, there are a total of 25 items used in this instrument. The instrument used in this study used an instrument that has been tested by in their research. Based on the results of these trials, the psychometric properties of the instrument have met the quality of a good instrument. This was indicated by the validity value for each item reaching more than .3 and the resulting reliability coefficient was also above .7. After that, the researchers distributed the questionnaires directly to the subjects who were the target of filling out with a duration of 2 weeks for the data collection. Data analysis was carried out by looking descriptively at the level of student satisfaction with the effectiveness of learning carried out by the tutors. The level of satisfaction was adjusted to the level of satisfaction above the average which indicates that if the results show a number above 75%, it means that the respondent is satisfied with the tutor's learning (Kanya et al., 2021). Meanwhile, correlational analysis using SPSS software was used to see the correlation and significance of the four aspects of learning carried out by the tutors, whether significant or not in supporting their learning performance. RESULTS AND DISCUSSION The implementation of data collection resulted in as many as 21 subjects who successfully completed the questionnaire until it was complete. These 21 subjects are the master students in English Education department in Open University who are still active. For demographic data, the researchers present it in table 1. Meanwhile, the results of the respondents' descriptive evaluation of the effectiveness of tutors in carrying out distance learning can be seen in table 2. Table 1. Respondent demographic data Category Total Percentage Gender Man 3 14.3% Woman 18 83.7% Home Address Bandung 19 91.5% Outside Bandung 2 9.5% Age 20 – 30 Years Old 3 14.3% 31 – 40 Years Old 15 71.5% Fauzy Rahman Kosasih, Juhana, Zulmi Ramdani, & Lidwina Felisima Tae English graduate student’s perception about the effectiveness of online learning tutor in distance education 4 41 – 50 Years Old 2 9.5% 51 – 60 Years Old 1 4.7% Occupation Language Teacher 21 100% Based on Table 1 regarding the data about research respondent, the majority of subjects in this study were dominated by female respondents. For the data on residence, most of them came from outside Bandung, which reached more than 90 percent. Meanwhile, the age of the subjects was dominated by the ages of 31-40 years which reached 15 people and the rest varied. For self- employment, all subjects worked as English teachers. Table 2. Result of descriptive evaluation on tutor effectiveness Aspect Item Average Total Average Percentage Reaction 1 3.76 3.51 87.75% 2 3.67 3 3.52 4 3.52 5 3.57 6 3.57 7 3.38 8 3.19 9 3.52 10 3.38 11 3.57 12 3.43 Learning 13 3.24 3.25 81.25% 14 3.19 15 3.33 Behavior 16 3.05 3.22 80.5% 17 3.19 18 3.24 19 31.4 20 3.33 21 3.38 Result 22 3.33 3.39 84.75% 23 3.62 24 3.14 25 3.48 Notes. The percentage results were obtained from the calculation of the total average value divided by the highest score on the scale, namely 4, then divided by 100%. The criterion is good if the percentage value is above 75%. The information in table 2 describes the description of the descriptive assessment given by the respondents to the existing tutors. It can be seen from table 2 that all the average values for the 25 items got the scores above 3 which means that interpretively the assessment given by respondents regarding the effectiveness of the tutors can be said to be good. Then, if we make a percentage of the total average value for each aspect, the reaction aspect got the largest percentage, which was 87.75%, and the behavior aspect got the lowest percentage, which was 80.5/5. However, in general all aspects were considered to be good because everything was in over 75%. In table 3 below, the researchers present the correlation results from each aspect that will support the effectiveness of distance learning itself (see table 3). Table 3. The correlation result of tutor effectiveness in distance learning Aspect Reacti on Learnin g Behavi our Result Reaction 1 .728** .717** .875** Learning .728** 1 .737** .754** Behaviour .717** .737** 1 .734** Result .875** .754** .734** 1 Notes: (The symbol ** represents a significant correlation at the 1% level) Referring to the table 3 described above, the aspect of correlation results shown to the tutor resulted in significant scores and were at a satisfactory stage. The correlation value between the four aspects tested showed the scores above .3, with a significance of .01. This of course has illustrated that there was a positive and significant relationship between the four aspects tested. This study has provided the information that was quite clear in explaining the evaluation given by the participants, in this case the master students of English Education department regarding the effectiveness of distance learning provided by the tutors. The researcher will in a structured way explain in advance the objectives to be explored in this study, which consist of two more specific objectives, namely explaining the description of the descriptive evaluation towards the quality of the tutors and further identifying the relationship between aspects in the assessment. The result has shown that the respondents felt that the existing tutors had provided good and effective teaching in the process. This can be seen from the high score of the assessment given to the tutors on the four aspects tested. The four aspects which included reaction, learning, behavior and results were considered good for the tutor. From the range of 0-100%, the percentage obtained in this study reached more than 75% for each aspect. These results certainly showed that the tutors had provided effective learning in these four aspects. When viewed from the explanation of each aspect, the reaction aspect was intended to see the satisfaction and interest in what has been shown by ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 11, Issue 1, February 2023 https://journal.uniku.ac.id/index.php/ERJEE 5 the tutors. Subjects considered that the tutors attracted enough attention in the learning process and what tutors did was able to attract and bind the students to be actively involved in various activities. In the learning aspect, the students also assumed that there was an increase in their knowledge and skills, one of which was due to the method and learning process carried out by the tutor. In terms of behavior, the result of tutor’s evaluation has shown that the behavior they showed and displayed was in accordance with what was learned with the tutor. As well as the last aspect, namely the results, in general, participants felt that the presence of a tutor could make them feel more satisfied and the final results that the students got were also according to their expectations. Thus, the assessment form of the tutor was very good and has shown that the tutor played an active role in distance learning for students in this study. Theoretically, the function and role of a tutor is of course being an educator as well as a partner for students to learn and discuss the challenging things (Fandiño & Velandia, 2020; Gould et al., 2014). In the distance learning scheme itself, being a tutor is not easy and it requires mastery of the latest technology and media so as to be able to support the existing distance learning process. Several previous studies also have assumed that the existence of a qualified tutor is able to provide an effective learning. In table 2 which has been explained previously, the reaction aspect got the highest score which was considered the best for the tutor. This reaction score was closely related to the level of care and creativity built by the existing tutor. This finding is also reinforced by the results of several previous studies which saw that personal aspects like this could be used as valuable capital in implementing the distance learning (Johansen, 2023; Obsuth et al., 2022; Palali et al., 2018). As it is known that the existence of distance learning always limits a student's communication with a tutor to be more limited and less open, so a tutor who is able to be communicative, interactive, and passionate about learning is needed (Fandiño & Velandia, 2020; Gould et al., 2014). Meanwhile, other results which can also be seen in table 3 further strengthen the researchers’ findings that these four aspects tested play an important role in the quality of the learning provided. All aspects were positively and significantly correlated. This shows that when a tutor has a high level in one aspect, it will also increase other aspects. Likewise, when one aspect is not optimal, the others will also not be optimal. As explained in the Kirkpatrick Model itself, which explains the four aspects as dimensions of continuity that will influence each other (Dewi & Kartowagiran, 2018; Farjad, 2012; Kirkpatrick & Kayser-Kirkpatrick, 2014; Sakthi & Moshi, 2021). In the first dimension, reaction and learning will be the two things that most determine how a tutor can be effective in providing learning. Meanwhile, in third dimension, the behavior aspect will be a manifestation of the previous dimension and in the last dimension, the result, will show the quality of the process that has been passed. In general, the result of this research has provided quite comprehensive information about the existence of a tutor in distance learning. The effectiveness of learning carried out by the tutors is essentially determined by many things, although one of the dominant factors in distance learning is the mastery of media and technology. The result of this study has indicated that the personal aspect is also considered to have an influence on the sustainability of distance learning itself. This research is of course very limited to the number of subjects which was only 21 people. However, the involvement of a number of respondents could become the good information because it can be used by the institution as the evaluation material, meanwhile for the tutors themselves, it is necessary to improve their personal aspects, including abilities in reaction, learning, behavior and results. Future research, of course, needs to involve more respondents by also looking more broadly at other non-personal factors such as relationships with students, organizational climate, and other technical factors. CONCLUSION This study has shown that the effectiveness of a tutor in implementing distance learning depends on his/her personal aspects. Respondents in this study assumed that the majority of existing tutors had carried out their roles effectively, judging by the assessment scores given by the respondents. ACKNOWLEDGEMENT The researchers thank the Research and Community Service Institute of the Universitas Terbuka for fully funding this study. We also thank all the respondents who have joined this study. REFERENCES Abdullah, W. (2018). Model blended learning dalam meningkatkan efektifitas Fauzy Rahman Kosasih, Juhana, Zulmi Ramdani, & Lidwina Felisima Tae English graduate student’s perception about the effectiveness of online learning tutor in distance education 6 pembelajaran. FIKROTUNA, 7(1), 855–866. https://doi.org/10.32806/jf.v7i1.3169 Alyoussef, I. Y. (2023). Acceptance of e-learning in higher education: The role of task- technology fit with the information systems success model. Heliyon, 9(3), e13751. https://doi.org/10.1016/j.heliyon.2023.e137 51 Abuhammad, S. (2020). Barriers to distance learning during the COVID-19 outbreak: A qualitative review from parents’ perspective. Heliyon, 6(11). https://doi.org/10.1016/j.heliyon.2020.e054 82 Affandi, M. R., Widyawati, M., & Bhakti, Y. B. (2020). Analisis efektivitas media pembelajaran e-learning dalam meningkatkan hasil belajar siswa SMA pada pelajaran fisika. Jurnal Pendidikan Fisika, 8(2), 150-157. https://doi.org/10.24127/jpf.v8i2.2910 Ardiasih, L. S., Emzir, E., & Rasyid, Y. (2018). Implementing constructivism approach in online writing course: learners need analysis. English Review: Journal of English Education, 6(2), 127-138. https://doi.org/10.25134/erjee.v6i2.1261 Budiastra, A. A. K., Warsihna, J., Widiasih, & Puspitasari, S. (2020). The development of science learning model for higher education of bachelor elementary school teacher education program in distance learning. Advances in Social Sciences Research Journal, 7(5), 54–69. https://doi.org/10.14738/assrj.75.8163 Charina, A., Kurnia, G., Mulyana, A., & Mizuno, K. (2022). Sustainable education and open innovation for small industry sustainability post covid-19 pandemic in Indonesia. Journal of Open Innovation: Technology, Market, and Complexity, 8(4), joitmc8040215. https://doi.org/10.3390/joitmc8040215 Churiyah, M., Sholikhan, S., Filianti, F., & Sakdiyyah, D. A. (2020). Indonesia education readiness conducting distance learning in covid-19 pandemic situation. International Journal of Multicultural and Multireligious Understanding, 7(6), 491– 507. https://doi.org/10.18415/ijmmu.v7i6.1833 Dewi, L. R., & Kartowagiran, B. (2018). An evaluation of internship program by using Kirkpatrick evaluation model. REiD (Research and Evaluation in Education), 4(2), 155–163. Dijk, D. van, & Schodl, M. M. (2015). Performance appraisal and evaluation. International Encyclopedia of Social & Behavioral Sciences, 17, 716–721. https://doi.org/10.1016/B978-0-08-097086- 8.22034-5 Escobar Fandiño, F. G., & Silva Velandia, A. J. (2020). How an online tutor motivates E- learning English. Heliyon, 6(8), e04630. https://doi.org/10.1016/j.heliyon.2020.e046 30 Farjad, S. (2012). The evaluation effectiveness of training courses in university by kirkpatrick model (Case Study: Islamshahr University). Procedia - Social and Behavioral Sciences, 46, 2837–2841. https://doi.org/10.1016/j.sbspro.2012.05.57 3 Foo, C. C, Cheung, B., & Chu, K. M. (2021). A comparative study regarding distance learning and the conventional face-to-face approach conducted problem-based learning tutorial during the COVID-19 pandemic. BMC Medical Education, 21(1), 1-6. https://doi.org/10.1186/s12909-021-02575- 1 Gould, D., Papadopoulos, I., & Kelly, D. (2014). Tutors’ opinions of suitability of online learning programmes in continuing professional development for midwives. Nurse Education Today, 34(4), 613–618. https://doi.org/10.1016/j.nedt.2013.06.006 Harrison, T. (2020). How distance education students perceive the impact of teaching videos on their learning. Open Learning: The Journal of Open, Distance and e- Learning, 35(3), 260–276. https://doi.org/10.1080/02680513.2019.170 2518 Johansen, K. (2023). Challenges regarding digital distance learning of operationally- oriented professions, due to Covid-19 pandemic. International Journal of Educational Research Open, 4, 100225. https://doi.org/10.1016/j.ijedro.2023.10022 5 Johnson, G. M., & Cooke, A. (2016). Self- regulation of learning and preference for written versus audio-recorded feedback by distance education students. Distance Education, 37(1), 107–120. https://doi.org/10.1080/01587919.2015.108 1737 ENGLISH REVIEW: Journal of English Education p-ISSN 2301-7554, e-ISSN 2541-3643 Volume 11, Issue 1, February 2023 https://journal.uniku.ac.id/index.php/ERJEE 7 Kang, K. I., Lee, N., & Joung, J. (2021). Nursing students’ experience of online peer tutoring based on the grow model: A qualitative study. Nurse Education Today, 107, 105131. https://doi.org/10.1016/j.nedt.2021.105131 Kanya, N., Fathoni, A. B., & Ramdani, Z. (2021). Factors affecting teacher performance. International Journal of Evaluation and Research in Education (IJERE), 10(4), 1462. https://doi.org/10.11591/ijere.v10i4.21693 Kirkpatrick, J., & Kayser-Kirkpatrick, W. (2014). The kirkpatrick for levels: A fresh look after 55 years. Kirkpatrick Partners. Lee, K., Choi, H., & Cho, Y. H. (2019). Becoming a competent self: A developmental process of adult distance learning. Internet and Higher Education, 41, 25–33. https://doi.org/10.1016/j.iheduc.2018.12.00 1 Luo, Y., Geng, C., Pei, X., Chen, X., & Zou, Z. (2021). The evaluation of the distance learning combining webinars and virtual simulations for senior nursing students during the covid-19 period. Clinical Simulation in Nursing, 57, 31–40. https://doi.org/10.1016/j.ecns.2021.04.022 Lytras, M. D., Serban, A. C., Ruiz, M. J. T., Ntanos, S., & Sarirete, A. (2022). Translating knowledge into innovation capability: An exploratory study investigating the perceptions on distance learning in higher education during the COVID-19 pandemic - the case of Mexico. Journal of Innovation & Knowledge, 7(4), 100258. https://doi.org/10.1016/j.jik.2022.100258 Martin, F., & Bolliger, D. U. (2018). Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning, 22(1), 205-222. https://doi.org/10.24059/olj.v22i1.1092 Martinez, F., Taut, S., & Schaaf, K. (2016). Evaluation classroom observation for evaluating and improving teaching: An international perspective. Studies in Educational Evaluation, 49, 15–29. https://doi.org/10.1016/j.stueduc.2016.03.0 02 Mishra, L., Gupta, T., & Shree, A. (2020). Online teaching-learning in higher education during lockdown period of COVID-19 pandemic. International Journal of Educational Research Open, 1, 100012. https://doi.org/10.1016/j.ijedro.2020.10001 2 Obsuth, I., Brown, R. H., & Armstrong, R. (2022). Validation of a new scale Evaluating the Personal, Interpersonal and Contextual dimensions of growth through learning – the EPIC scale. Studies in Educational Evaluation, 74, 101154. https://doi.org/10.1016/j.stueduc.2022.1011 54 Palali, A., van Elk, R., Bolhaar, J., & Rud, I. (2018). Are good researchers also good teachers? The relationship between research quality and teaching quality. Economics of Education Review, 64, 40–49. https://doi.org/10.1016/j.econedurev.2018.0 3.011 Prasojo, L. D., Fatmasari, R., Nurhayati, E., Darmadji, A., Kusumaningrum, F. A., & Andriansyah, Y. (2019). Indonesian state educational universities’ bibliometric dataset. Data in Brief, 22, 30–40. https://doi.org/10.1016/j.dib.2018.11.128 Ramdani, Z., Jaenudin, U., Djamal, N. N., Anggara, A., Pertiwi, F., Syamsiyah, I., Luthfi, M., Juliana, P., & Wisda, T. (2021). The false memory and the effect of murottal al-Qur’an. Jurnal Psikologi Islami, 7(1), 1–9. Ramdani, Z., Kembara, M. D., Alhapip, L., Amri, A., Warsihna, J., & Anas, Z. (2021). Teachers’ perception and readiness towards adaptive learning in the covid-19 pandemic: Thematical content analysis study. AL- ISHLAH: Jurnal Pendidikan, 13(2), 1383– 1393. https://doi.org/10.35445/alishlah.v13i2.915 Ramdani, Z., Marliani, R., & Rahman, A. A. (2019). The individual work performance scale: A psychometric study and its application for employee performance. Humanities and Social Sciences Reviews, 7(5), 405–414. https://doi.org/10.18510/hssr.2019.7545 Riyanto, A., & Yunani, E. (2020). The effectiveness of video as a tutorial learning media in muhadhoroh subject. Akademika, 9(02), 73–80. https://doi.org/10.34005/akademika.v9i02.1 088 Rosyadi, S. (2018). Revolusi industri 4.0 peluang dan tantangan bagi alumni universitas terbuka. Universitas Jenderal Soedirman. Saidi, R. M., Sharip, A. A., Abd Rahim, N. Z., Zulkifli, Z. A., & Md Zain, S. M. (2021). Evaluating students’ preferences of open and distance learning (odl) tools. Procedia Fauzy Rahman Kosasih, Juhana, Zulmi Ramdani, & Lidwina Felisima Tae English graduate student’s perception about the effectiveness of online learning tutor in distance education 8 Computer Science, 179, 955–961. https://doi.org/10.1016/j.procs.2021.01.085 Sakthi, T., & Moshi, A. A. M. (2021). Effectiveness measurement study on entrepreneurship awareness camp for technical education students using Kirkpatrick’s model. Materials Today: Proceedings, 45, 1609–1613. https://doi.org/10.1016/j.matpr.2020.08.419 Suárez, J. L., García, S., & Herrera, F. (2021). A tutorial on distance metric learning: Mathematical foundations, algorithms, experimental analysis, prospects and challenges. Neurocomputing, 425, 300–322. https://doi.org/10.1016/j.neucom.2020.08.0 17 Vinogradova, I., & Kliukas, R. (2015). Methodology for evaluating the quality of distance learning courses in consecutive stages. Procedia - Social and Behavioral Sciences, 191, 1583–1589. https://doi.org/10.1016/j.sbspro.2015.04.36 4 Warsihna, J., M Anwas, E. O., Anas, Z., Kosasih, F. R., Ramdani, Z., & Amri, A. (2021). Kesiapan guru dalam mengimplementasikan buku panduan pembelajaran pasca bencana. Kwangsan: Jurnal Teknologi Pendidikan, 9(2), 142-152. https://doi.org/10.31800/jtp.kw.v9n2.p142-- 152