International Journal of Interactive Mobile Technologies (iJIM) – eISSN: 1865-7923 – Vol. 15, No. 19, 2021 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by Teachers https://doi.org/10.3991/ijim.v15i19.24837 Thaer A.R. Abukhalil1, Shaima M-F. Halawani1, Wajeeh Daher1,2() 1Arab American University, Ramallah, Palestine 2An-Najah National University, Nablus, Palestine wajeehdaher@gmail.com Abstract—The study examined the criterions that school principals used to evaluate the effectiveness of the use of distance learning tools in the COVID- 19 pandemic by school teachers. Six school administrators were interviewed regarding their evaluation of the technological tools used by the teachers in distance. The constant comparative method was used to analyze the interview data. The study results indicated ten criterions that served the principals in their evaluation:(1) Availability of the technological tool in students’ hands, (2) The teacher’s ability to handle the tool used, (3) The fit of the used tool to the specific content, (4) The fit of the used tool to the curriculum objectives, (5) Planning the use of the tool in the classroom, (6) Use of technological tools for interac- tive communication in the classroom, (7)Use of technological tools to stimulate, attract and engage students in distance learning, (8) Use of technological tools to achieve quality education, (9) Use of technological tools to manage students’ behavior in distance learning, and (10) Using mobile tools to encourage students’ engagement in distance learning. More research is needed to study school princi- pals’ behavior regarding E-learning especially in emergency education. Keywords—distance learning, e-learning, the COVID-19 pandemic, school principals, technological tools 1 Introduction Educational institutions invest in technology-enhanced learning in order to ensure the success of this learning [1], which raises the question of how this learning can be shaped to be as effective as possible. The need to employ e-learning tools in ‘Pre Covid-19’ education had been debated. However, with the spread of the epidemic situ- ation of Covid-19 pandemic, the E-learning environments became essential for the con- tinuation of educational processes as students were requested to stay at home and limit the spread of the virus. Schleicher [2] point out that since March 2020, the world has been witnessing an event that threatened education with a crisis that may have been the most dangerous in our contemporary time, and this event has led to a global educational crisis. In this global educational crisis, many students could not receive the basic skills they need in real life (ibid). This is especially true in low- and middle-income countries, as Palestine, even before the Covid-19 pandemic. 64 http://www.i-jim.org https://doi.org/10.3991/ijim.v15i19.24837 mailto:wajeehdaher@gmail.com Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… The World Bank index showed that the percentage of students who could read at the age of 10 is 53% in low- and middle-income countries just before the outbreak of the Corona virus, and the pandemic had made that outcome even worse [3]. One way to overcome the results of the Coruna pandemic is to use e-learning, where during this pandemic, E-learning tools became an integral part of the entire educational process in terms of education using modern technology such as multimodal resources and tools. It is interesting to examine the school administrators’ perspective on the effectiveness of E-learning tools during educational emergencies as Covid-19. The current study seeks to further this issue. 1.1 Literature review Basilaia and Kvavadze [4] describe E-Learning as an organized process that aims to achieve educational outcomes using technological means that utilize sound, image, film, and interaction between the learner, content, and educational activities. Koumi [5] considers e-learning as the result of rapid technological developments, especially after the educational process has been directly affected by the automation of industry, the development of industrial intelligence technology, the Internet of things, and the infor- mation technology revolution that broke into the classroom and became an integral part of it. Since the advent of computers, countries have developed plans for informatics, making electronic technologies part of educational curricula. They integrated technol- ogy in education to reduce traditional learning problems and to accelerate students’ advancement in knowledge and to enable more people to return to learning. Covid-19 pandemic as escalator of E-learning in the world in general and Palestine in particular. In a report on education under the Corona pandemic, De Giusti [6] noted that the Corona pandemic Covid-19 had created the largest disruption of educational systems in history. Closures of schools and higher education institutions affected 94% in low- and middle-income countries. The crisis reduced the opportunities for many children, young people, and adults in the most vulnerable groups of poor, rural, girls, and refugees (ibid). On the other hand, the crisis led to innovation within the education sector in support of the continuity of education and training by utilizing different technological means, including distance education, radio, and television. Institutions tried different means for online learning as the utilization of various disruptive community learning resources [7] and mobile learning [8]. Espino-Díaz et al. [9] say that the spread of Covid-19 resulted in a qualitative shift that has occurred in the world around the learning and education methodologies. This shift was represented in the movement of most countries around the world from traditional learning to e-learning using online applications, media, and programs that facilitate the learner’s interaction with educational content. These online applications were expected to help students at all levels of education, build knowledge. Many coun- tries have resorted to educational emergency under the Covid-19 pandemic. Irfan, Kusumaningrum, Yulia and Widodo [10] reported that during the Covid-19 Pandemic, universities in Indonesia have implemented various online learning policies. Doing so, they confronted several challenges experienced by lecturers and students. These challenges included: limitations in presenting material, especially when the material included scientific equations. Besides, the lecturers’ use of technological tools was iJIM ‒ Vol. 15, No. 19, 2021 65 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… limited to PowerPoint, finding difficulty in using effectively video editing. In addi- tion, Draissi and ZhanYong [11] reported that the Moroccan universities, under calls from the Ministry of Education of Morocco, have responded reasonably quickly by implementing online education network platforms to help instructors manage their instruction through appropriate training. Palestine, like other countries that moved to e-learning, also tried to be fast moving. Schools received increased pressure to incorporate e-learning tools into curricula. Palestine declared a state of emergency under the spread of the pandemic, in which it resorted to E-learning. The Ministry of Education has activated distance learning strategies and techniques to cope with the state of emergency called by the Palestin- ian government. In more detail, the Palestinian Ministry of Education has activated distance learning strategies such as Zoom platforms, Teams, a television channel for learning. All the previous means were intended to fit all educational levels. Researchers also attempted to study the educational benefits from the Pandemic. Basilaia and Kvavadze [4] describe part of Georgia’s educational experience during the Covid-19 pandemic, saying that that this experience benefited teachers, students and school administration in developing technology skills and learning materials that could be utilized in the post-pandemic period. All these experiences during the Pandemic made teachers develop new perspective of distance learning, which could benefit teach- ers in the post-pandemic period. Effectiveness of teaching in general and E-learning in particular. Researchers were interested in the effectiveness in teaching in general. Money [12] referred in his study to a range of factors that help the teacher achieve efficiency in teaching as knowledge of subject matter, ability to communicate effectively, ability to motivate, being friendly and open, having well organized material, having the ability to control classroom conduct, and ability to inspire interest in course material. This ability to communicate, especially with the teacher, was also emphasized by researchers as means for effective online learning [ex., 13]. During Covid-19 pandemic, researchers studied the conditions for the effectiveness of the e-learning experience in the educational emergency. Yulia [14] say that e-learn- ing can be effective if teachers adopt a learning design around an educational mate- rial that effectively meets goals, examine students’ learning needs, identify goals and means to achieve them, choose measurement and feedback tools, and choose appropri- ate learning methods. Yulia [14] argue that the e-learning can be enriched by choosing the appropriate educational software for communication, which serves effective and widespread communication among students. The authors say that electronic learning and teaching could suffer from poor assessment reliability, difficult testing control, and poor monitoring to avoid cheating. These shortages could be overcome if teachers use formative assessment during interaction with students. Van Nuland et al. [15] suggested to promote e-learning by using simulations, tools that support learners’ understanding of models and processes, so that these tools help the educational process achieve efficiency, effectiveness, and satisfactory outcomes. A good use of e-learning tools keeps users’ time, and makes possible the easiness to plan, coordinate, organize, organize content, and familiarize students with the material. Many aspects of the use tool used in e-learning should be considered in terms of its availability and its easiness for use for both the instructor and the student. 66 http://www.i-jim.org Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… Efficient tools can be found in all operating systems and mobile devices. They need to have a high degree of security and access, be effectively and constructively aligned with the curriculum, and provide the skills needed to achieve quality for the teacher and the instructor. Efficient tool can be used to create interactive content, study plans, and organize tasks, and provide clear and easy assessments. In their attempts to study effectiveness of e-learning, researchers used various approaches. Panigrahi et al. [16] examined the effectiveness of e-learning by considering the impact of stu- dent engagement on perceived learning effectiveness in the context of Indian higher education. Further, they studied, through the social cognitive theory, the impact of personal factors, as Internet self-efficacy and environmental factors, on various dimensions of student engagement; specifically, behavioral, emotional and cognitive aspects. The results of the study indicated that the environmental factors) positively impacted the perceived learning effectiveness. The Internet self-efficacy affected the perceived learning effectiveness through the mediating effect of all the dimensions of student engagement. Furthermore, there existed a positive relationship between the perceived learning effectiveness and student achievement. Aljaser [17] found that the e-learning environment is effective in developing academic attainment in learning English among fifth grade students. Ogbonna [18] reported that patterns of synchronous and asynchronous interaction in e-learning impacted positively the computer skills of secondary students. Role of school administrators in online learning. Researchers attempted to investigate the role of school administrators in online learning. Valentine [19] identified five main challenges to distance learning that administrators must deal with: quality of instruction, costs effectiveness, misuse of technology, role of technicians, and problems with equipment. McFarlane [20] considered the quality of instruction, misuse of technology, and costs effectiveness the most critical among the five to distance education and thus administrators should deal with. Quilici and Joki [21] investigated the roles of online school administrators. Doing so, they examined six sets of paired online administrators and teachers who interacted in a supervision/ evaluation cycle. The findings showed that the administrators viewed themselves as instructional leaders, while the teachers viewed their online administrators as administrators. Hashim, Kayode and Hassan [22] investigated distance learning administrators of the three selected universities manage, monitors and improve communication and inter- actions between distance learning instructors and learners to promote self-regulated learning. The results of the study showed methods used by the administrators (1) suit- able teaching qualification, (2) monitoring of forum activities (3) feedbacks (4) taking care of learning platform, and (5) training. Murphy [23] investigated the role of the administrators in implementing blended learning in Algebra I courses in South Carolina public schools. The results showed two primary roles for the administrator: Administrator as technology leader and adminis- trator as administrator. The primary behaviors of the administrators were communi- cating a strong vision as well as developing a plan to look after the vision. The major goals of the administrators were to improve student achievement and to make available diversity for struggling students. Little research has been done on how administrators evaluate teachers’ use of technological tools in distance education. The present study attempts to do so. iJIM ‒ Vol. 15, No. 19, 2021 67 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… 1.2 Research rationale and goals In 2020, the Palestinian Ministry of Education sought to provide facilities for e-learning through a range of programs such as the E-school and Teams programs, with the aim of supporting, promoting and developing E-learning in COVID-19 educational emergency. To assist teachers in their online instruction, the ministry conducted a series of technology-related training programs to improve the performance and quality of E-learning for teachers, supervisors and all those working in E-learning during the period of the Covid-19 pandemic. One issue related to this e-learning experience is how school administrators evaluate the use of tools by the school teachers. This issue is related to the claim of Quilici and Joki [21] that online administrators cannot simply demand innovation from their teachers to meet changing student needs and different government accountabilities; they have to lead the innovation. Online administrators also must not only know about online learning, but they have to invest in online learn- ing, and they have to guide their teachers to adapt their teaching to online methods and tools. One way to do so is to monitor and evaluate the teachers’ choice of techno- logical tools to regulate this use. The present study intends to investigate how admin- istrators of Palestinian public schools evaluated teachers’ use of technological tools in the COVID-19 Pandemic. The present study is a preliminary one as it verifies the evaluation performed by six administrators. 1.3 Research question What criteria are used by public school administrators to evaluate the use of techno- logical tools by school teachers during the COVID-19 Pandemic? 2 Methodology 2.1 Research context and participants In the frame of COVID-19 educational emergency, the Ministry of Education in Palestine has been working on the dissemination of online educational activities and les- sons through training educational supervisors and teachers in programs that address the use of distance education tools in online teaching. Doing so, it utilized electronic platforms that broadcast lessons with the help of channels like TV channels and YouTube channels. Platform like Moodle, E-school and Teams began to be utilized as means of communica- tion for educational purposes. School administrators were expected to monitor and reg- ulate the use of technological tools by school teachers during their online teaching. The present research attempts to verify school administrators’ implementation of technological tools in online learning in the schools by interviewing six high school administrators. The administrators were chosen on condition that their schools were reported, by the ministry of education, to have implemented successfully e-learning during Covid-19 Pandemic. 2.2 Data collection tools The data was collected through semi-structured interviews with the participants, where each interview lasted 45–60 minutes. The interview protocol consisted of 68 http://www.i-jim.org Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… open-ended questions that were used to extract in-depth responses from participants’ experience. The participants agreed to record the interviews. A consent form was signed by the participants. The questions were repeated or, if necessary, reworded, until the interviewer was convinced that the interviewee had fully expressed the thought she had shared. The previous precautions were performed taking into consideration that the interviewees might not understand the wording of the questions (e.g., metaphors) or perceive different meanings of the questions; not those intended by the interviewer. The interviewees were, therefore, encouraged to ask questions during the interview in order to be sure of the requirement of the questions. The interview questions were: 1. What criteria did you use to evaluate the tools that the school teachers used to in e-learning? 2. How did make sure that the teachers effectively used the tools for content purposes? 3. How did make sure that the teach- ers effectively used the tools for communication purposes? 4. How did make sure that the teachers effectively used the tools to motivate, attract and engage students in their learning? 5. How did make sure that the teachers effectively used the tools to achieve a qualitative evaluation of the students’ online learning? 6. How did make sure that the teachers effectively used the tools to effectively manage the students’ online behaviour? 2.3 Data analysis tools We used the deductive and inductive constant comparative method [24] to analyze the interviewees’ answers to the interview questions. Doing that, we followed works that utilized these methods before (e.g., [25–26]). Validity and reliability. To ensure the validity and reliability of the study, the researchers discussed the themes and sub-themes before approving the final themes. To ensure reliability and validity of data analysis, each researcher worked alone, transcribed the interviews and observation, and coded them. The three authors analyzed 25% the transcriptions of the interviews. The inter-rater reliability between the coders was initially 71.42%. After the three researchers discussed and negotiated the results of the coding, the inter-rater reliability among the coders became 85.71%. 3 Study results Answering the research questions, we will address the categories mentioned by the principals as criteria that serve them to evaluate teachers’ use of technological tools in their online teaching. 3.1 Availability of the technological tool in students’ hands It was important for the six administrators to evaluate the teacher’s choice of a tech- nological tool by examining its availability in the hands of the students. Salem; one of the administrators noted: “the technological tool must be available to students. It would iJIM ‒ Vol. 15, No. 19, 2021 69 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… not help if the teacher were creative and the technological tool was not available to students, because his or her effort will be in vain.” 3.2 The teacher’s ability to handle the tool used Administrators stated that the teachers should be able to use distance learning tool in order to use effectively the tool in their teaching. Alaa; one of the administrators said: “I made sure that the teacher knows how to handle the tool, for he or she would not uti- lize the tool effectively if he or she is not in control of the technical issues of the tool.” 3.3 The fit of the used tool to the specific content Principals emphasized the importance of following up on teachers’ procedural plans to suit distance learning tools to content. Alaa said: “teachers need to prepare the content in a manner that attracts the student, the planning of the quota, and the setting of quota targets. The first condition for doing that is to fit the technological tool for the specific content.” 3.4 The fit of the used tool to the curriculum objectives Principals agreed on the importance of providing a technological tool that fits the achievement of the curriculum objectives of the learning material. Samira, one of the administrators said: “Technological tools need to fit the curriculum objectives. I encouraged the teachers to take care of this issue and told them I would take that into consideration when evaluating their use of technological tools.” 3.5 Planning the use of the tool in the classroom Principals stressed the need to define plans, to prepare the course in advance and to define its objectives, since this preparation has an impact on students’ scientific experi- ence. Salem said: “I request my teachers to send me their plans of the lessons in which they use technological tools. Planning is a condition for effective teaching.” All principals also made it clear that planning and content design help make the curriculum suitable for e-learning, which make e-learning more successful in support- ing students’ achieving of goals. 3.6 Use of technological tools for interactive communication in the classroom Principals agreed on the importance of communication between the teacher and students and between the students themselves. They stressed that the communication process should continue effectively through technological tools, as in the case of face- to-face meetings. Interactive communication was of great importance in the educational process for the principals. Ahlam said: “Principals need to encourage the activation of technological tools in distance learning as it contributes to the strengthening of active relationships in the school community.” 70 http://www.i-jim.org Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… 3.7 Use of technological tools to stimulate, attract and engage students in distance learning It was important for principals that teachers use technological tools which has the potential, not only to motivate and attract students, but also to encourage them to participate actively in educational events. Samira said: “One additional criterion for evaluating teachers’ use of technological tools in the COVID19 pandemic was the ability of the tools to promote students’ attraction and engagement. This will ensure the outcome of students’ participation in distance learning.” 3.8 Use of technological tools to achieve quality education School principals agreed that it was important to use different tools to achieve quality education, including quality evaluation. Alaa said: “Technological tools, as forums, can provide means for students to share their views with themselves and with their teachers. Other tools, as the Padlet, can provide means for interaction, and so on. I evaluated teachers’ use of tools looking at their utilization of tools to guarantee quality education as communication and interaction.” Ahlam said: “it is necessary for the student to feel that there is a real evaluation in the distance learning process. The teacher should follow up on the recruitment of a part of the student’s participation and interaction in the evaluation process. To do so, there is need to use various tools and methods such as group projects, educational calendar tasks and the electronic completion file.” Salem said: “I evaluated the school teachers regarding their utilization of the various options in E-meet and Google classroom, espe- cially the evaluation options.” All the principals above considered the variety of tools as important to ensure the quality of teaching and learning in distance learning. For them, the variety of tools leads to the enrichment of the educational process, especially its evaluation component. 3.9 Use of technological tools to manage students’ behavior in distance learning Principals stressed the importance of monitoring the behavior of students in distance learning. Samira said: “It is the role of the principal to guide teachers regarding their monitoring of students’ behavior. This is especially true in distance learning because it is a new type of learning.” Amir said: “It is important for teachers to use a variety of tools that support them in their management of students’ behavior in distance learning. In fact it is more a strategy than a tool. One such strategy is to record the lesson and put the recording in the classroom site. This would make students pay more attention to their behavior as it is recorded. The teachers in our school were requested to do so. They were evaluated taking this issue into consideration.” 3.10 Using mobile tools to encourage students’ engagement in distance learning Principals stressed the importance of using tools in mobile devices. Salem said: “I encouraged the teachers in the school to use the tools that fit mobile devices. I told my teachers that I will appreciate their use of such devices. We know that more students iJIM ‒ Vol. 15, No. 19, 2021 71 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… now have mobile devices, so they would be willing to engage in distance learning that utilize these devices.” 4 Discussion The present research intended to study the criterions used by school principals in their evaluation of the technological tools used by teachers in online teaching. The research results indicated that there are ten criterions that served the school principals in doing that. Below we discuss each criterion and its meanings. 4.1 Availability of the technological tool in students’ hands The first criterion followed by the school principals was the availability of the technological tool in students’ hands. It seems that this availability is a must, since without this availability, students could not benefit from the use of the technological tool. Moreover, this criterion is in line with researchers consideration of the availability of technological tools in the classroom as a primary condition for the utilization of this tools [27]. Specifically, Ertmer [28] considered the availability of technology tools and resources a main barrier to technology integration. This consideration indicates the necessity of the availability of the technological tool in students’ hands as one of the main conditions for a successful utilization of the tool. 4.2 The teacher’s ability to handle the tool used The second criterion followed by the school principals was the teacher’s ability to handle the tool used. This criterion could be considered a complimentary condition of the first one since there are four participants in the classroom regarding tool’s use: the tool itself, the teacher who chooses the tool for students’ learning, the student who uses the tool, and the content that the tool addresses. Furthermore, the centrality of the teacher’s technological knowledge in implementing technology in the classroom is mentioned by Çoklar and Yurdakul [29] who say that teachers are providers of educa- tional sustainability, so they need to adapt themselves to the developing technologies applicable to learning environments, which is connected with technology integration. 4.3 The fit of the used tool to the specific content The third criterion used by the schools’ principals to evaluate teachers’ use of tech- nological tools is the fit of the used tool to the specific content. It connects two of the main participants in technology-based learning, namely tool and content. This fit is considered necessary for characterizing an activity that is based on the tool [30]. 4.4 The fit of the used tool to the curriculum objectives The fourth criterion used by the schools’ principals to evaluate teachers’ use of tech- nological tools addresses another fit, that of the used tool to the curriculum objectives. This fit was one of the corner stones of a practical tool for teachers, developed by 72 http://www.i-jim.org Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… Mukherjee [31] to assess classroom teaching and learning with technologies. One cri- terion for this assessment was effectiveness that addressed “how the technology sup- ported effective pedagogies and supported content conceptual development and the attainment of lesson objectives.” (p. 109) 4.5 Planning the use of the tool in the classroom The fifth criterion used by the schools’ principals to evaluate teachers’ use of tech- nological tools addresses planning the use of the tool in the classroom. This concern of the principals with planning the use of the tool in the classroom is understood as good teachers are expected to be concerned with lesson planning [32]. This specifically true when talking about teaching online lessons as part of an emergency education. More- over, this is in line with researchers’ pointing at planning as having a main role in the success of technology integration in the school (e.g., [33]). 4.6 Use of technological tools for interactive communication in the classroom The sixth criterion used by the schools’ principals to evaluate teachers’ use of tech- nological tools addresses the use of technological tools for interactive communication in the classroom. This concern of the principals with the interactive communication in the classroom meets researchers’ concern with developing technological tools that encourage the interaction in the classroom. Eastman [34] argues that interactive tech- nology can help teachers enhance communication in the classroom, so there is more attention and interest in the content by the students. Moreover, Goncharenko [35] suggests developing interactive technologies based on the active student-student and student-teacher interaction and communication. 4.7 Use of technological tools to stimulate, attract and engage students in distance learning The seventh criterion used by the schools’ principals to evaluate teachers’ use of technological tools addresses the use of technological tools to stimulate, attract and engage students in distance learning. The use of this criterion indicates principals’ awareness of the potentiality of technological tools to contribute to students’ affective learning. This potentiality was also emphasized by researchers. Miarso [36] argues that technological tools can stimulate students’ feelings, thoughts, willingness, and atten- tion. Puspitarini and Hanif [37] says that a variety of technological tools can foster stu- dents’ engagement in learning activity, which would increase their motivation to learn. 4.8 Use of technological tools to achieve quality education The eighth criterion used by the schools’ principals to evaluate teachers’ use of technological tools addresses the use of technological tools to achieve quality educa- tion. This concern of the principals with tools potentiality for quality education mean that this quality is assumed by the principals to be one potentiality of technological tools. Abuzant et al. [38] found that distance learning platforms, specifically Google iJIM ‒ Vol. 15, No. 19, 2021 73 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… Classroom, could serve as a quality learning environment. In addition, quality educa- tion could be achieved through electronic manipulatives, as they facilitate the problem solving done by students [39]. Specifically, electronic manipulatives could be used in mobile technologies to contribute to the different aspects of students’ learning [40]. 4.9 Use of technological tools to manage students’ behavior in distance learning The ninth criterion used by the schools’ principals to evaluate teachers’ use of tech- nological tools addresses the use of technological tools to manage students’ behavior in distance learning. This criterion in fact could be considered a result of some of the previous criterions as the use of technological tools to stimulate, attract and engage stu- dents in distance learning. Teachers could manage students’ behavior in distance learn- ing when they attract them to this learning. In addition, managing students’ behavior in distance learning is one of the concerns of distance learning platforms [41]. This con- cern implies the importance educational institutions put on managing students’ behavior in online learning. Here, this management emerged as important for school principals. 4.10 Using mobile tools to encourage students’ engagement in distance learning The tenth criterion used by the schools’ principals to evaluate teachers’ use of tech- nological tools addresses the use of mobile tools. The principals considered this use to encourage students’ engagement in distance learning. This use is not only able to encourage students’ engagement in their learning [42], even in in COVID 19 situations, but also their enjoyment of this learning [43]. In addition, mobile-based problem-solving skills for students were higher than in the case of regular problem solving [44]. Conclusions and recommendations. The present research aimed to examine the criterions used by school principals in their evaluation of the technological tools used by teachers in online teaching. The research results indicated that there are ten criterions that served the school principals in doing that. These criterions could be related to each of the components centric for teaching and learning in the classroom, especially in the online classroom: the student, the teacher, the curriculum and the tool. Availability of tools in the hands of students was one criterion, while the expertise of teachers was a second criterion. Some criterions were related to the quality and effectiveness of teaching, while one criterion was related to using mobile devices in online learning to make learning online more accessible to students [45–47]. Future research needs to study how principals consider the contribution of digital tools to the social aspect of students’ learning, as this aspect would support students’ cognitive aspect [48], affective aspect [49] and behavioral aspect [50] of students’ learning. 5 References [1] Wall, J. (2015). Leadership in implementing technology-enhanced learning in educational institutions. E-Learning-Instructional Design, Organizational Strategy and Management, 393–413. https://doi.org/10.5772/61108 74 http://www.i-jim.org https://doi.org/10.5772/61108 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… [2] Schleicher, A. (2020). The impact of covid-19 on education insights from education at a glance 2020. Paris, France: OECD Publishing. [3] Azevedo, J. P., Hasan, A., Goldemberg, D., Iqbal, S. A., & Geven, K. (2020). Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. The World Bank. https://doi.org/10.1596/1813-9450-9284 [4] Basilaia, G., & Kvavadze, D. (2020). Transition to online education in schools during a SARS-CoV-2 coronavirus transition to online education in schools during a SARS-CoV-2 coronavirus (COVID-19) pandemic in Georgia. Pedagogical Research, 5(4). em0060. https://doi.org/10.29333/pr/7937 [5] Koumi, J. (2006). Designing video and multimedia for open and flexible learning. New York, NY: Routledge. https://doi.org/10.4324/9780203966280 [6] De Giusti, A. (2020). Policy brief: Education during COVID-19 and beyond. Revista Iberoamericana de Tecnología En Educación y Educación En Tecnología, 26, e12–e12. https://doi.org/10.24215/18509959.26.e12 [7] Hardika, H., Aisyah, E., & Listyaningrum, R. (2021). Utilization of various disruptive community learning resources for the Covid-19 period in the perspective of life based learning. International Journal of Interactive Mobile Technologies (IJIM), 15(07), 123–139. https://doi.org/10.3991/ijim.v15i07.21551 [8] Pebriantika, L., Wibawa, B., & Paristiowati, M. (2021). Adoption of mobile learning: The influence and opportunities for learning during the Covid-19 pandemic. International Journal of Interactive Mobile Technologies (IJIM), 15(05), 222–230. http://dx.doi.org/ 10.3991/ijim.v15i05.21067 [9] Espino-Díaz, L., Fernandez-Caminero, G., Hernandez-Lloret, C. M., Gonzalez-Gonzalez, H., & Alvarez-Castillo, J. L. (2020). Analyzing the impact of COVID-19 on education professionals. toward a paradigm shift: ICT and neuroeducation as a binomial of action. Sustainability, 12(14), 5646. https://doi.org/10.3390/su12145646 [10] Irfan, M., Kusumaningrum, B., Yulia, Y., & Widodo, S. A. (2020). Challenges during the pandemic: Use of e-learning in mathematics learning in higher education. Journal of Mathematics Education, 9(2), 147–158. [11] Draissi, Z., & ZhanYong, Q. (April 27, 2020). COVID-19 Outbreak Response Plan: Imple- menting Distance Education in Moroccan Universities. Available at SSRN: https://ssrn.com/ abstract=3586783 or http://dx.doi.org/10.2139/ssrn.3586783 [12] Money, S. M. (1992). What is teaching effectiveness? A survey of student and teacher per- ceptions of teacher effectiveness. ERIC 1992-9/6 Reports Research (143). [13] Daher, W., & Awawdeh Shahbari, J. (2020). Secondary students’ identities in the virtual classroom. Sustainability, 12(11), 4407. https://doi.org/10.3390/su12114407 [14] Yulia, H. (2020). Online learning to prevent the spread of pandemic corona virus in Indonesia. ETERNAL (English Teaching Journal), 11(1). https://doi.org/10.26877/eternal.v11i1.6068 [15] Van Nuland, S. E., Hall, E., & Langley, N. R. (2020). STEM crisis teaching: Curriculum design with e-learning tools. FASEB BioAdvances, 2(11), 631–637. https://doi.org/10.1096/ fba.2020-00049 [16] Panigrahi, R., Srivastava, P. R., & Panigrahi, P. K. (2020). Effectiveness of e-learning: The mediating role of student engagement on perceived learning effectiveness. Information Technology & People. https://doi.org/10.1108/ITP-07-2019-0380 [17] Aljaser, A. M. (2019). The effectiveness of e-learning environment in developing academic achievement and the attitude to learn English among primary students. Turkish Online Jour- nal of Distance Education-TOJDE, 20(2), 176–194. https://doi.org/10.17718/tojde.557862 [18] Ogbonna, C. G. (2016). effects of synchronous and asynchronous e-learning modes on stu- dents’ achievement and skill acquisition in word processing in secondary schools in Nsukka Local Government Area of Enugu State (Unpublished Master thesis). University Of Nigeria, Nsukka, Nigeria. iJIM ‒ Vol. 15, No. 19, 2021 75 https://doi.org/10.1596/1813-9450-9284 https://doi.org/10.29333/pr/7937 https://doi.org/10.4324/9780203966280 https://doi.org/10.24215/18509959.26.e12 https://doi.org/10.3991/ijim.v15i07.21551 http://dx.doi.org/10.3991/ijim.v15i05.21067 http://dx.doi.org/10.3991/ijim.v15i05.21067 https://doi.org/10.3390/su12145646 http://dx.doi.org/10.2139/ssrn.3586783 https://doi.org/10.3390/su12114407 https://doi.org/10.26877/eternal.v11i1.6068 https://doi.org/10.1096/fba.2020-00049 https://doi.org/10.1096/fba.2020-00049 https://doi.org/10.1108/ITP-07-2019-0380 https://doi.org/10.17718/tojde.557862 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… [19] Valentine, D. (2002). Distance learning: Promises, problems, and possibilities. Online Jour- nal of Distance Learning Administration, 5(3). https://www.westga.edu/~distance/ojdla/ fall53/valentine53.html [20] McFarlane, D. A. (2011). The leadership roles of distance learning administrators (DLAs) in increasing educational value and quality perceptions. Online Journal of Distance Learning Administration, 14(1). https://www.westga.edu/~distance/ojdla/spring141/McFarlane141. html [21] Quilici, S. B., & Joki, R. (2011). Investigating roles of online school principals. Journal of Research on Technology in Education, 44(2), 141–160. https://doi.org/10.1080/15391523.2 011.10782583 [22] Hashim, C. N., Kayode, B. K., & Hassan, S. S. S. (2015). The roles of administrators in distance education programme: A case at higher learning institutions. International Journal of Social Science and Humanity, 5(5), 479. https://doi.org/10.7763/IJSSH.2015.V5.503 [23] Murphy, C. J. (2017). Role of the principal in implementing blended learning in algebra I courses in South Carolina public schools (Unpublished Ph. D. dissertation). University of South Carolina, Carolina, USA. [24] Glaser, B., & Strauss, A. (1967). The Discovery of Grounded Theory: Strate- gies for Qualitative Research. London, UK: Weidenfeld and Nicolson. https://doi. org/10.1097/00006199-196807000-00014 [25] Shahbari, J. A., & Daher, W. (2020). Learning congruent triangles through ethnomathemat- ics: The case of students with difficulties in mathematics. Applied Sciences, 10(14), 4950. https://doi.org/10.3390/app10144950 [26] Baya‘a, N., Daher, W., & Anabousy, A. (2019). The development of in-service mathematics teachers’ integration of ICT in a community of practice: Teaching-in-context theory. Inter- national Journal of Emerging Technologies In Learning (IJET), 14(01), 125–139. http:// dx.doi.org/10.3991/ijet.v14i01.9134. [27] Francom, G. M. (2016). Educational Technology Use among K-12 Teachers: What Tech- nologies Are Available and What Barriers Are Present? https://files.eric.ed.gov/fulltext/ ED582680.pdf [28] Ertmer, P. A. (1999). Addressing first-and second-order barriers to change: Strategies for technology integration. Educational technology research and development, 47(4), 47–61. https://doi.org/10.1007/BF02299597 [29] Çoklar, A. N., & Yurdakul, I. K. (2017). Technology integration experiences of teachers. Discourse and Communication for Sustainable Education, 8(1), 19–31. https://doi.org/ 10.1515/dcse-2017-0002 [30] Park, C. (2019). Exploring a new determinant of task technology fit: Content characteristics. Journal of International Technology and Information Management, 27(3), 100–118. [31] Mukherjee, M. M. (2013). Technological tools for science classrooms: choosing and using for productive and sustainable teaching and learning experiences (Unpublished Doctoral dissertation). The University of Queensland, Australia. [32] Jensen, L. (2001). Planning lessons. In M. Celce-Murcia (ed.), Teaching English as a Second or Foreign Language, 403–408. Boston, MA: Heinle & Heinle. [33] Al-Hunaiyyan, A., Alhajri, R., & Bimba, A. (2021). Towards an efficient integrated distance and blended learning model: How to minimize the impact of COVID-19 on education. International Journal of Interactive Mobile Technologies (IJIM), 15(10), 173–193. doi: https://doi.org/10.3991/ijim.v15i10.21331 [34] Eastman, J. K. (2007). Enhancing classroom communication with interactive technology: How faculty can get started. College Teaching Methods & Styles Journal (CTMS), 3(1), 31–38. https://doi.org/10.19030/ctms.v3i1.5273 [35] Goncharenko, N. (2018). Interactive technologies of teaching Russian as a foreign lan- guage for medical students. Medical University, 1(1), 40–43. https://doi.org/10.2478/ medu-2018-0006 76 http://www.i-jim.org https://www.westga.edu/~distance/ojdla/fall53/valentine53.html https://www.westga.edu/~distance/ojdla/fall53/valentine53.html https://www.westga.edu/~distance/ojdla/spring141/McFarlane141.html https://www.westga.edu/~distance/ojdla/spring141/McFarlane141.html https://doi.org/10.1080/15391523.2011.10782583 https://doi.org/10.1080/15391523.2011.10782583 https://doi.org/10.7763/IJSSH.2015.V5.503 https://doi.org/10.1097/00006199-196807000-00014 https://doi.org/10.1097/00006199-196807000-00014 https://doi.org/10.3390/app10144950 http://dx.doi.org/10.3991/ijet.v14i01.9134 http://dx.doi.org/10.3991/ijet.v14i01.9134 https://files.eric.ed.gov/fulltext/ED582680.pdf https://files.eric.ed.gov/fulltext/ED582680.pdf https://doi.org/10.1007/BF02299597 https://doi.org/10.1515/dcse-2017-0002 https://doi.org/10.1515/dcse-2017-0002 https://doi.org/10.3991/ijim.v15i10.21331 https://doi.org/10.19030/ctms.v3i1.5273 https://doi.org/10.2478/medu-2018-0006 https://doi.org/10.2478/medu-2018-0006 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… [36] Miarso, Y. (2009). Sowing educational technology. Jakarta: Kencana Prenada Media Group. [37] Puspitarini, Y. D., & Hanif, M. (2019). Using learning media to increase learning motivation in elementary school. Anatolian Journal of Education, 4(2), 53–60. https://doi.org/10.29333/ aje.2019.426a [38] Abuzant, M., Ghanem, M., Abd-Rabo, A., & Daher, W. (2021). Quality of using google classroom to support the learning processes in the automation and programming course. International Journal of Emerging Technologies in Learning, 16(6), 72–87. https://doi. org/10.3991/ijet.v16i06.18847 [39] Daher, W. (2009). Preservice teachers’ perceptions of applets for solving mathematical prob- lems: Need, difficulties and functions. Journal of Educational Technology & Society, 12(4), 383–395. [40] Baya’a, N., & Daher, W. (2010). Middle school students’ learning of mathematics using mobile phones: Conditions and consequences. Journal of Interactive Learning Research, 21(2), 165–185. [41] Amer, A., & Daher, W. (2019). Moodle quizzes as a teaching tool in English for academic purposes course. International Journal of Innovation and Learning, 25(1), 35–49. https:// doi.org/10.1504/IJIL.2019.096513; https://doi.org/10.1504/IJIL.2019.10016636 [42] Miao, F., Huang, R., Liu, D., & Zhuang, R. (2020). Ensuring effective distance learning during COVID-19 disruption: Guidance for teachers. Paris, France: UNESCO. [43] Ali, M. M., Mahmood, M. A., Anjum, M. A. I., & Shahid, A. (2020). The acceptance of mobile assisted language learning as primary learning tool for learners in COVID 19 situations. PalArch’s Journal of Archaeology of Egypt/Egyptology, 17(12), 382–398. [44] Amin, A., Sudana, I., Setyosari, P., & Djatmika, E. (2021). The Effectiveness of mobile blended problem based learning on mathematical problem solving. International Journal of Interactive Mobile Technologies (IJIM), 15(01), 119–141. https://doi.org/10.3991/ijim. v15i01.17437 [45] Qureshi, M., Khan, N., Hassan Gillani, S., & Raza, H. (2020). A systematic review of past decade of mobile learning: What we learned and where to go. International Journal of Interactive Mobile Technologies (IJIM), 14(06), 67–81. doi: https://doi.org/10.3991/ijim. v14i06.13479 [46] Hanafi, Y., Murtadho, N., & Ikhsan, M. A. (2020). Reinforcing public university student’s worship education by developing and implementing mobile-learning management system in the ADDIE instructional design model. International Journal of Interactive Mobile Technol- ogies, 14(2), 215–241. https://doi.org/10.3991/ijim.v14i02.11380 [47] Ivanova, R., Ivanov, A., & Nikonova, Z. (2020). Application of mobile technologies in for- eign language learners’ project activity. International Journal Of Interactive Mobile Tech- nologies (IJIM), 14(21), 64–77. doi: https://doi.org/10.3991/ijim.v14i21.18471 [48] Daher, W., Anabousy, A. & Jabarin, R. (2018). Metacognition, positioning and emotions in mathematical activities. International Journal of Research in Education and Science (IJRES), 4(1), 292–303. https://doi.org/10.21890/ijres.383184 [49] Baya’a, N., & Daher, W. (2014). Facebook as an Educational Environment for Mathematics Learning. In G. Mallia (Ed.), The Social Classroom: Integrating Social Network Use in Education (pp. 171–190). IGI Global. http://doi:10.4018/978-1-4666-4904-0.ch009 [50] Daher, W., & Baya’a, N. (2012). Characteristics of middle school students learning actions in outdoor mathematical activities with the cellular phone. Teaching Mathematics and its Applications: An International Journal of the IMA, 31(3), 133–152. https://doi.org/10.1093/ teamat/hrr018 iJIM ‒ Vol. 15, No. 19, 2021 77 https://doi.org/10.29333/aje.2019.426a https://doi.org/10.29333/aje.2019.426a https://doi.org/10.3991/ijet.v16i06.18847 https://doi.org/10.3991/ijet.v16i06.18847 https://doi.org/10.1504/IJIL.2019.096513 https://doi.org/10.1504/IJIL.2019.096513 https://doi.org/10.1504/IJIL.2019.10016636 https://doi.org/10.3991/ijim.v15i01.17437 https://doi.org/10.3991/ijim.v15i01.17437 https://doi.org/10.3991/ijim.v14i06.13479 https://doi.org/10.3991/ijim.v14i06.13479 https://doi.org/10.3991/ijim.v14i02.11380 https://doi.org/10.3991/ijim.v14i21.18471 https://doi.org/10.21890/ijres.383184 http://doi:10.4018/978-1-4666-4904-0.ch009 https://doi.org/10.1093/teamat/hrr018 https://doi.org/10.1093/teamat/hrr018 Paper—School Principals’ Evaluation of the Effectiveness of Employing Distance Learning Tools by… 6 Authors Thaer A Abukhalil is a Ph.D. student in the Educational Administration Depart- ment at the Arab-American University in Ramallah, Palestine. He works in Palestinian ministry of education as the head of the Field Affairs Department. Shaima M-F Halawani is a Ph.D. student in the Educational Administration department at the Arab-American University in Ramallah, Palestine. She works in the Examination Institute at the Hebrew University of Jerusalem, Israel. She is interested in research that studies team work, administration and leadership. Wajeeh Daher is a Professor Doctor with the Department of Educational Sciences, An-Najah National University and the Head of the MTeach Program with the Al-Qasemi Academic College of Education. He has authored and coauthored numerous articles in the field of mathematics education and the field of technology in education. His articles address the different aspects of students’ learning and pre-service teachers’ education Article submitted 2021-06-17. Resubmitted 2021-09-08. Final acceptance 2021-09-08. Final version published as submitted by the authors. 78 http://www.i-jim.org