Ukraine higher education based on data-driven decision making (DDDM) CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 Ukraine higher education based on data-driven decision making (DDDM) Kateryna Bondar1,2, Olena Shestopalova2, Vita Hamaniuk2 and Vyacheslav Tursky2 1Freie Universität Berlin, Habelschwerdter Allee 45, Berlin, 14195, Germany 2Kryvyi Rih State Pedagogical University, 54 Gagarin Ave., Kryvyi Rih, 50086, Ukraine Abstract. This article presents a theoretical review and empirical study of the assessment of the quality of BA and MA online education during the Russian military invasion of Ukraine in 2022. A qualitative theoretical analysis and comparison are made with a study of higher education quality assessment across Ukraine during the war in 2022 (N = 12019). The article analyzes the tools and structure of student feedback on the evaluation of the educational process. The undergraduate and graduate programs are modified based on the analysis of this data. In the Ukrainian higher education space, the National Qualifications Framework requirements for each specialty have been under development since 2014. The National Qualifications Framework developed a list of interdisciplinary competencies that should be transferred to academic courses. According to the Ukrainian higher education standard, universities must provide empirical evidence that their students actually acquire these competencies. Baseline data on the assessment of the quality of online education by student teachers during the war in Ukraine is presented using the case study of the Kriviy Rih State Pedagogical University (N = 688). Further modifications of the questionnaire to assess the quality of teaching in order to improve data-driven decision-making and testing ethics are proposed. Keywords: course assessment, self-assessment of competencies, higher education, data-driven decision- making, online learning, quality of education, quality of online learning, ethic of testing, quality of teachers’ activity 1. Background context In the context of the invasion of the Russian army into the territory of Ukraine and the annexation of Ukrainian territories since February 2022 [18], supporting the quality of higher education of citizens is one of the priorities of the Ukraine as a guarantee of future economic stability [10, 11]. Accordingly, one of the priorities in the war conditions of hostilities is to preserve the quality of higher education in Ukraine. An investigation of students’ attitudes towards higher education in 2022 was commissioned by the Friedrich Naumann Foundation for Freedom. Information " katerynabondarr@gmail.com (K. Bondar); e.shestopalova@kdpu.edu.ua (O. Shestopalova); vitana65@gmail.com (V. Hamaniuk); tursky.kdpu@gmail.com (V. Tursky) ~ https://www.ewi-psy.fu-berlin.de/einrichtungen/arbeitsbereiche/ewi-psy/Mitarbeitende/Aktuelles-Team/ Bondar_K/index.html (K. Bondar); https://kdpu.edu.ua/personal/opshestopalova.html (O. Shestopalova); https://kdpu.edu.ua/personal/vagamanuk.html (V. Hamaniuk); https://kdpu.edu.ua/personal/votursky.html (V. Tursky) � 0000-0002-2441-4203 (K. Bondar); 0000-0002-3401-1790 (O. Shestopalova); 0000-0002-3522-7673 (V. Hamaniuk); 0000-0003-1033-922X (V. Tursky) CTE Workshop Proceedings © Copyright for this paper by its authors, published by Academy of Cognitive and Natural Sciences (ACNS). This is an Open Access article distributed under the terms of the Creative Commons License Attribution 4.0 International (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 346 https://doi.org/10.55056/cte.564 mailto:katerynabondarr@gmail.com mailto:e.shestopalova@kdpu.edu.ua mailto:vitana65@gmail.com mailto:tursky.kdpu@gmail.com https://www.ewi-psy.fu-berlin.de/einrichtungen/arbeitsbereiche/ewi-psy/Mitarbeitende/Aktuelles-Team/Bondar_K/index.html https://www.ewi-psy.fu-berlin.de/einrichtungen/arbeitsbereiche/ewi-psy/Mitarbeitende/Aktuelles-Team/Bondar_K/index.html https://kdpu.edu.ua/personal/opshestopalova.html https://kdpu.edu.ua/personal/vagamanuk.html https://kdpu.edu.ua/personal/votursky.html https://orcid.org/0000-0002-2441-4203 https://orcid.org/0000-0002-3401-1790 https://orcid.org/0000-0002-3522-7673 https://orcid.org/0000-0003-1033-922X https://acnsci.org/cte https://creativecommons.org/licenses/by/4.0 https://acnsci.org CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 on 26 parameters was gathered; for example, level of education by gender, state of residence (internal migration, migration abroad, residence), the format of university education (online synchronous/asynchronous, hybrid, in classrooms, suspended), and the content of education. According to the study “Higher Education in the Eyes of Students” (N = 12019), 94.75% of students are aimed at obtaining higher education, and only 5.25% of university students affirmatively answered that education is meaningless [30]. This indicates the priority of the quality of education for students. It should be noted that according to all-Ukrainian data, teaching at universities resumed mainly in online format (73.34%). In particular, students answered that the university education was resumed in online format synchronously (55.8%), resumed in online format asynchronously (18.06%), resumed in mixed online format (synchronously and asynchronously) (23.25%) and other options – 3.41% (resumed, part of the classes are conducted online, and part – in person in the classrooms, the educational process is suspended) [30]. In response to the need for Ukrainian universities to realize the online format of learning in the condition of the invasion of the Russian army into the territory of Ukraine and the annexation of Ukrainian territories in February 2022, all universities had to shift to Multi-access learning [16] and HyFlex learning [22]. However, it remains unclear how effective teaching and learning have been in this ”trial mode” of synchronous online courses compared to face-to-face or even asynchronous online formats [22]. According to Beatty [3], “the format is HyFlex (hybrid + flexible), which explains the key principles of HyFlex courses as (a) allowing students to choose how to attend classes, (b) offering equivalent learning activities in all forms, (c) using the same learning objects for all students, (d) providing students with technology and skills to participate in all forms, and (e) using authentic assessments. According to its proponents, the HyFlex model is more student-centered and more flexible than standard mixed-mode classrooms because students can choose how to adapt their learning needs to the course environment”. This creates a request for crisis management and decision-making based on the analysis of the quality of online teaching in wartime. As evidenced by the answers to the question “I think more often than before about whether the right specialty I have chosen” – only 25.03% have doubts about the chosen profession. At the same time, only 7.2% answered that “At the first opportunity I will change my specialty”. External motivation in studying is observed in 10.82% of students (“If it were not for the insistence of my parents, I would have already quit studying”). Everything indicates that it is especially important for students to receive a quality education as a condition for their stable future, on the one hand. On the other hand, for teachers, there is a need for flexible transformation of teaching to ensure a high level of education. To assess the quality of online teaching in wartime, it is especially important to develop sensitive tools for feedback. At the same time, these evaluation tools should be convenient for processing large amounts of feedback data and optimizing the learning process. This creates the first research question: Is multiple types of testing methods (valid, reliable, and fair) for assessing the quality of teaching in order to improve data-driven decision-making and ethics in a crisis situation? The Center for Innovative Development of Higher Education of the National Academy of Educational Sciences of Ukraine and the Center for Social and Humanitarian Research of 347 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 V. N. Karazin Kharkiv National University implemented a large-scale project “Students of Higher Education Institutions about their studies” during two months of the 1st semester of 2021 using an anonymous online survey, in which more than 2.7 thousand bachelor’s and master’s students of 39 higher education institutions in all regions of Ukraine took part. The parameters of the sample meet the requirements of representativeness, which allows extrapolation of the findings to the general population of students of higher education institu- tions in Ukraine. Almost two-thirds (62%) are satisfied with the level of education quality, and accordingly, one-third (33%) are dissatisfied with only certain aspects of education. In particular, dissatisfaction is caused by teaching methods using Internet communications and computer technologies [29]. At the same time, the majority of students note that in the educational process, teachers apply the latest methods – electronic presentations, use Internet content, online services, video conferencing, etc. To organize training, university teachers use various means of digital communication. At the same time, 30% of students point to the obsolescence of the material, and the lack of new scientific information in the curricula, including 25% of students being dissatisfied with the attitude of teachers towards students, and the level of available communication. The survey shows that the most common online classes are conducted using the Zoom platform, less often – Google Meet, and communication between teachers and students often takes place in groups using the two most popular applications – Viber and Telegram. Moodle [25] and Google Classroom [33] learning management systems are used to organize the distance educational process, manage it and ensure interaction between teachers and students. Saukh et al. [29] points to the urgent need to improve the digital competence of teachers, which should not be reduced to setting tasks and checking their implementation. It is argued that universities should develop a specific program of psychological support for participants in the educational process and the transition to various forms of distance learning. The second research question is: What are the ways of anti-crisis digital transformation of teaching at the university in the conditions of military operations based on decision-making based on survey data? 2. Objectives 2.1. Educational data-driven decision-making Today, data-driven decision-making in higher education is a component of goal-setting in university management regarding curricula, choice of subjects, and types of teaching to improve student performance, course recommendation, analysis of learning patterns, prediction of dropout, enhancing the efficiency of teachers, and reducing administrative burden [28]. There are several uncomfortable issues for data-driven decision-making in higher education (figure 1). Data collection is possible with the use of a number of educational systems and platforms, which are reviewed in the article “Big Educational Data & Analytics: Overview, Architecture and Challenges” by Ang, Ge and Seng [2], who note that: “Educational data is rapidly being collected and generated in the higher education ecosystem, which encompasses various systems and platforms such as course management and learning management systems (LMS), massive open online courses (MOOCs), OpenCourseWare (OCW), open educational resources (OERs), 348 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 Figure 1: Uncomfortable issues for data-driven decision-making in higher education [35]. and social media sites such as Twitter, Facebook, YouTube, and personal learning environments (PLEs). Learning Management Systems (LMS) are education management platforms for admin- istering, delivering, tracking and reporting on curricula and courses. Moodle is one of the most popular open source LMS options available today. Other examples of LMS are Canvas, Sakai, ATutor, Eliademy. Forma LMS, Dokeos and OpenOLAT. The concept of LMS originated from e-learning”. According to a review by Ang, Ge and Seng [2], one of the challenges every institution faces is how to improve student retention and reduce dropout (figure 2). The authors point to two emerging areas for improving student retention and reducing dropouts through data analysis: dropout prediction; and the development of academic early warning systems. 2.2. Data-driven decision-making in higher education in Ukraine: mechanisms of state regulation Data-driven decision-making in Higher education is a relevant and rapidly developing area in Ukraine. In particular, in accordance with the policy of digitalization of the state in 2022- 2023, the platform “Digital Transformation of Higher, Professional Pre-Higher and Professional (Vocational) Education (e-University)” is being developed on an ongoing basis in Ukraine. The aim of the project is to automate the admission campaign, organize the recruitment and training (internship) of foreigners and stateless persons, order educational documents and European- style supplements to them, introduce electronic licensing, modernize the Unified State Electronic Database on Education, create and modernize a unified electronic system for monitoring the employment of graduates [26]. 349 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 Figure 2: Reasons for student dropout. In turn, the service includes electronic licensing in the field of education (e-licensing), which is based on the work of a separate body for evaluating the work of the educational institution – the National agency for higher education quality assurance. According to the Strategy of the National agency for higher education quality assurance to 2022, one of the goals is to harmonize standards and develop criteria for ensuring the quality of higher education based on best international and national practices [23]. According to the Strategy for the Development of Higher Education [23], it provides for the protection of higher education from academic dishonesty, the provision of low-quality educational services, lack of interaction and trust between stakeholders (figure 3). To solve this problem, it is advisable to use Data-driven decision-making and educational data mining (EDM) – interrelated areas in educational research. The National agency for higher education quality assurance provides recommendations on the views on the current state of testing the quality of higher education and typical questionnaires for online surveys. However, there is currently no comprehensive study or literature review on models of questionnaires and grading scales for the Ukrainian higher education system. One of the tasks of the reform of higher education in Ukraine transforms the structure and mechanisms of student feedback on the educational process. Bachelor’s and Master’s degree programs are created on the basis of the analysis of these data, which gives the right to employment. In the space of Ukrainian higher education, there has been the development of 350 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 Figure 3: Model questions for assessment [23]. requirements for the National Qualifications Framework of each specialty since 2014. For the National Qualifications Framework, a list of interdisciplinary competencies must be transferred into academic courses. According to the Ukrainian Higher Education Standard, universities must provide empirical evidence that their students are actually acquiring these competencies [15]. If we analyze the mechanism proposed by the National agency for higher education quality assurance (methodology of independent, external, on-site evaluation of the quality of legal education in Ukraine), the questionnaire was implemented from the American educational system and contains an assessment of the university’s policy on the quality of evaluation of educational, scientific and educational-professional programs and should include a generalized assessment of the uniqueness of the program using qualitative analysis (focus groups) proposed by the administration, faculty, and employers and quantitative (statistical) quality assessment. This survey consists of two parts. In the first section, respondents rated each statement on a 5-point Likert scale from “(1) never true” to “(5) almost always true” and indicated to what extent they agreed with it to what extent you agree with it according to the monitoring blocks in figure 3 (54 questions). Section 2 contains general comments “The main strengths of my school are”, “The main weaknesses of my school are” [23]. At this stage of educational practices in Ukraine, educational programs of Ukrainian uni- versities do not yet have such unified standardized assessment tools and evidence base for analysis in the form of scientific publications, on the one hand. On the other hand, there is a nationwide mechanism of mandatory evaluation, in accordance with the standard procedure for licensing educational programs for university students, conducted by the National agency for higher education quality assurance every 4-5 years for each specialty. Thus, in the Ukrainian 351 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 educational space it is relevant to create a scientifically based research standardized algorithm for monitoring the quality of teaching disciplines and the type of motivation for choosing disciplines by students and postgraduates. 2.3. Data-driven decision-making in university management: research Analyzing the tools to improve student retention and reduce dropout through data analysis and learning analytics presented in figure 2, it should be emphasized that the Model Question- naire (testing) of the Strategy of the National agency for higher education quality assurance to 2022 (presented in figure 3) refers to the analysis of academic problems, financial prob- lems and psychosocial problems. However, it requires theoretical understanding and practical implementation of testing the assessment of the motivational component in data collection. The work “Konstruktion eines Fragebogens zur Erfassung der Lehrkompetenz (LeKo)” by Thiel, Blüthmann and Watermann [34] presents the technique of constructing and validating questionnaires to determine the level of teaching competence. It was developed at the Freie Universität Berlin as a universal tool for teaching evaluation without any connection to the subject. The article examines different approaches to teaching evaluation. It is emphasized that there are several approaches to this process, each of which represents a certain perspective of vision: from the position of experts, from the position of students, and from the position of the teacher himself (self-evaluation). Evaluating teaching competence from any perspective has both advantages and disadvantages compared to the other two, so the quality of the evaluation depends primarily on the choice of an effective validation tool, one of which, according to the authors, is the educational success of the students. In addition, for an adequate evaluation, the researchers emphasize, both areas of competence (Kompetenzbereiche) and quality criteria and indicators (Qualitätsdimensionen) must be clearly defined [34]. Based on the analysis of scientific research of previous years, the generalization of already developed forms of questionnaires for evaluating the quality of education, the authors of the article proposed the form of the questionnaire developed by them for determining the level of teaching competence, theoretically substantiated and experimentally tested its effectiveness [34]. In the structure of the methodology “Das Berliner Evaluationsinstrument für selbsteingeschätzte, studentische Kompetenzen (BEvaKomp)” conducted by Braun et al. [7], contains a questionnaire of 29 items consisting of six subscales, measuring, success in knowledge acquisition, methodol- ogy/presentation, communication/collaboration competence and personal competence. The results show sufficient reliability, and factor analysis proves that the instrument establishes differences between competencies and has sufficient discriminant validity. Two ideas went into the methodology for creating the instrument: 1. Why are measures of subjective competence important in assessing the educational success of a course of study? 2. How can subjective competency acquisition be recorded as a result of attending a course? According to Braun et al. [7], in order to assess the contribution of individual courses to subjective competency acquisition visibly, students are asked to rate the increase in competency as a result of attending the course. 352 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 A distinction is made between professional competence, methodological competence, social competence, and personal competence. When applied to university education, the four areas of competence mean the following: 1. Professional competence means that students expand their knowledge, understanding, application, and analytical skills [27]. 2. Methodological competence refers to an individual’s ability to effectively plan work and master appropriate work methods [17]. 3. Social competence refers to the ability to realize one’s own goals while protecting the interests of others [6]. Some authors distinguish it in communication and cooperation skills [36]. 4. Personal competence describes an individual’s productive attitude toward learning and self-development. The authors focused on these four domains of competence in developing an instrument to measure subjective skill acquisition while attending university courses. A detailed definition of the different competency domains can be found in [7]. The main conclusions of the validation of the method are that, according to the Bologna Process, course evaluations should be based on competencies and not – as has been almost exclusively the case so far – on student satisfaction with the teaching staff. The tool developed differs significantly from other assessment tools. This adaptive approach is an extension of Staufenbiel and Hartz [32], who developed different scales for course types (lectures, seminars, exercises). Thus, a descriptive assessment of course- level filter questions also allows conclusions to be drawn about the extent to which courses of an interactive nature (seminars, exercises) enable the development of social skills. For example, the evaluation criterion for seminars can be the number of students’ oral presentations (descriptive evaluation of the filter question on communication skills) [8]. The development of BEvaKomp is understood as the first step toward outcomes-based and competency-based course assessment, as required by university reforms under the Bologna Process. BEvaKomp records self-assessment in non-specialized competency areas. Future studies will need to investigate the extent to which subjective competencies can be validated using “more rigorous criteria” such as academic performance and professional success after graduation [8]. In the study by Braun et al. [7], Braun and Leidner [8], a systematic analysis of content and construct validity was carried out to create the methodology “Assessment in Higher Education: Self-Assessment of Competencies (HEsaCom)”. The methodological difference of the methodol- ogy from similar ones is that students evaluate different aspects of their growth and benefits from the course, not how much they liked the teacher. Braun and Leidner [8] acknowledge that when using the self-report method, it is impossible to use strict standardization criteria, as well as the fact that students’ self-assessment of teaching quality can be distorted by students’ liking or disliking of the teacher, the grades they received in the course. The authors rightly believe that leveling this kind of distortion is possible if students self-assess the growth of their own cross-cutting competencies. When developing HEsaCom, the authors conducted an analysis of construct validity, in particular, differentiating between 353 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 the construct of self-assessment of competence acquisition and the construct of teaching and teacher assessment. The general concept of the methodology was the hypothesis that it is able to selectively measure the acquisition of competencies according to students’ own assessments as opposed to the assessment of teaching quality. Confirmatory factor analysis (CFA) was conducted to hypothetically distinguish between the constructs of competence acquisition and course satisfaction. The obtained factor models made it possible to substantiate how the HEsaCom methodology differs from a similar questionnaire by Westermann et al. [38], which is focused on the teacher’s personality and assessment of his/her professional behavior. Among the scales of the HEsaCom methodology are “Acquisition of knowledge”, “Systematic competence”, “Presentation competence”, “Communication competence”, “Competence in cooperation” and “Personal competence” [7]. The ideas of Bloom [5], Braun and Clarke [9], Dochy and Alexander [12], Erpenbeck [14], Wentzel [37] concepts were used for the meaningful validation of individual scales. The factor analysis conducted by the researchers indicates that the structure of the association of the six competency areas with academic course satisfaction is confirmed by the global and local CFA fit indices [4, 21], and thus it is established that the acquisition of cross-cutting competencies by students and their satisfaction with teaching are different constructs. Thus, an important conclusion of the design and testing of the methodology is that the highest priority of the criterion of quality of higher education such as the acquisition of competencies is approved. Therefore, the assessment of pedagogical behavior and teacher’s personality is a less important criterion. The main theoretical contribution and novelty of the HEsaCom methodology is that it allows for the evaluation of the course focused on the result and competence. The use of such a criterion allows, according to Braun et al. [7], to encourage teachers to think more about the results of students than about their own pedagogical behavior and it provides an opportunity to introduce a culture of result orientation. In particular, Dübbers and Schmidt-Daffy [13] proposes a thematic survey intervention to encourage students’ self-motivation. The study indicates that course instructors and experts in DBDM (Motivation: Non-regulation, External regulation, Introjected regulation, Identified regulation, Integrated regulation, Intrinsic regulation). Thus, exam-related reasons were formed into items measuring external regulation, as exams are often a strong and visible external influence on university students. Items representing more independent forms of motivation were formed (e.g., “I studied because this content will be useful for me as a future teacher”), and items representing less independent forms of motivation (e.g., “I prepared for the final knowledge test to get confirmation of active participation in the lecture”). The scores were integrated into an overall score, the so-called Relative Autonomy Index: RAI = (internal regulation + integrated regulation + identified regulation) – (introjected regulation + external regulation motivation). This concise and well-validated measure allows comparing differences in the degree of self-determination of students’ motivation with a single score, with higher scores indicating more self-determined motivation in relation to less self-determined motivation [13]. Also, ethics regarding the content of the survey should be observed. For example, in the study by Piskurska [24] on the types of monitoring of the quality of education, the author aptly notes that, first of all, most questionnaires assess the quality of a teacher’s work and focus 354 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 on the personal qualities and behavior of the teacher. The author proposes an algorithm for monitoring the quality of education in the form of 6 stages of research. Also, the researcher rightly notes that based on the data of student surveys to demand changes in the personality of the teacher is a violation of human rights, but it is quite legitimate for the administration of a higher education institution to change the work of the teacher (facilitation skills, student- centered learning, methods of stimulating individual work). Given this circumstance, the objects of monitoring in the form of questions can only be those requirements for the organization of the educational process, which are fixed in the regulatory documents of each particular higher education institution: regulations on the organization of the educational process, job descriptions of teachers, regulations on the current control of student performance and interim certification, etc. Also, Piskurska [24] emphasizes that the content of monitoring should include only those requirements, the implementation of which is likely to be problematic. In another study on the significance and complexity of test tasks during the evaluation of the performance of university teachers, an analysis of existing approaches to the construction of rating systems, educational standards and curricula are proposed to generalize theoretical questions of the essence, structure, functions, features of the application of test control of student learning success during the evaluation of professional activities of university teachers. The publication confirmed the possibility of evaluating the teacher’s pedagogical activity based on the results of monitoring the success of student’s studies and developed a method for calculating the importance of a test task without an expert [1]. Machova, Komarkova and Lnenicka [20] and others in their study proposed a cloud archi- tecture for analyzing educational data from the Moodle system in the cloud using Apache Hadoop. This study also presents data mining in an e-learning environment from an educational perspective. 3. Research methods 3.1. Methods In the pilot (N = 62) we used a questionnaire for online surveys proposed by the National agency for higher education quality assurance (Methodology of independent, external, on-site evaluation of the quality of legal education in Ukraine). The questionnaire was implemented from the American educational system. This survey consists of two parts. In the first section, respondents rated each statement on a 5-point Likert scale from “(1) never true” to “(5) almost always true” and indicated to what extent they agreed with it to what extent you agree with it according to the monitoring blocks in figure 3 (54 questions). Section 2 contains general comments “The main strengths of my school are”, “The main weaknesses of my school are” [23]. However, the proposed questionnaire on assessing the quality of teaching subjects in military and distance learning environments proved to be unreliable when piloted in April 2022 because students 1) did not complete it to completion (32%), 2) ignored the test (16%), and 3) answered the same way on a 5-point scale with a score of 5 (27%), thereby creating a positive feedback gala (as indicated by the high test completion rate: 4-6 minutes) without realizing the questions asked. Therefore, it was decided to modify the questionnaire to highlight the most important 355 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 indicators for the university consisting of 20 questions on a 5-point scale, 3 questions with answer options, and 1 open-ended question. For all psychological variables, respondents gave answers on a 5-point Likert scale from “(1) never true” to “(5) almost always true” in Google Forms. The expediency of using Google Forms was dictated by the restriction of access to computers and the possibility of using the phone in connection with the evacuation of students from the occupied territories of Ukraine by Russia. Issues that pointed to the successful Design of Programs: 1.1. I need discipline for my future professional activity. 1.2. The discipline contains useful material. 1.3. The discipline is logically connected with other disciplines. 1.4. The discipline contributes to the formation of skills and abilities. 1.20. In general, it was interesting for me to master this discipline. In this study, every student was asked questions that indicated the style of teaching (Teaching and Assessment): 1.6. The professor is fluent in educational material and modern scientific information; 1.7. The professor motivates students to independently search for information in depth; 1.8. The professor clearly formulates the goals and the training plan; 1.9. The educational material is presented in an accessible and interesting way; 1.10. The professor uses the latest interactive teaching methods. The possible benefits from Student-centered learning were evaluated with the following questions: 1.12. I have the desire to continue studying with this teacher (other disciplines, coursework, qualification work); 1.13. The teacher is open and friendly with students; 1.14. I always had the opportunity to turn to the teacher for clarification or advice (items: 1.12-14) and The teacher is tactful and knows how to establish contact with students (item: 1.16). The questions that indicated the Learning Resources and Student Support, Certification: 1.5. The discipline is provided with the necessary textbooks and teaching materials. 1.11. The teacher clearly defines the criteria for assessing students’ knowledge. 1.17. The professor always conducts classes on time and according to the schedule. 1.18. Distance learning was well organized by the professor. 1.19. But, in my opinion, it would be more correct to teach the discipline personally. The Cronbach’s alpha of this scale (Teaching and Assessment) was 0.69, minimum 1, maximum 5, a range of 1.12, variance of 0.20, and a mean of 4.37. Test–retest reliability with an average interval of 2 weeks showed a Pearson’s correlation of r = 0.71 for the Design of Programs, r = 0.79 for the Teaching and Assessment, r = 0.67 for Student-centered learning, and r = 0.72 – Learning Resources and Student Support, Certification. Perceived benefits related to studying goals were then examined using 4 items that were asked to assess the difficulty of the academic discipline (question 2). Is discipline contributed to a specific studying goal such as a clearer professional self-image, or increased professional hard skills? 3. Estimate the ratio of the amount of educational material and the time allocated for its processing. 4. When did the teacher provide you with information about the procedure and evaluation criteria? The internal consistency of these items was 0.82 measured with Cronbach’s alpha. The test-retest reliability with an average interval of 2 weeks for each item/each treatment goal varied from 0.51 to 0.70 measured with Spearman’s rho (𝜌). These items were analyzed on a single-item level because we wanted to explore each goal by itself and therefore were not considered as a scale. 356 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 3.2. Data analysis methods We plan to realize our design of research on the quality of teaching in Universities in war conditions. The respondents’ perceptions of the status of the quality of teaching are analyzed based on four indicators: (1) Style of teaching (Teaching and Assessment), (2) Student-centered learning (3) the Learning Resources and Student Support, Certification, and (4) Design of Programs. The survey of the quality of teaching in Universities in war conditions (computer-based surveys powered by Google forms) contained 20 questions and 4 statements about perceived benefits from courses, associated factors, and participants of studying process behavioral characteristics. These answers were scored on a 5-point Likert scale from “(1) never true” to “(5) almost always true”. These items were analyzed on a single-item level because we wanted to have a specific look at the content of each item. Data analysis will be analyzed using SPSS 20. Generally, type of analyses is conducted: descriptive statistics (means and standard deviations) were used to identify the level of explained variables (1) Style of teaching (Teaching and Assessment), (2) Student-centered learning (3) the Learning Resources and Student Support, Certification, and (4) Design of Programs. 3.3. Participants We assessed 122 academic courses in 11 teacher preparation with on areas (kindergarten teachers, elementary school teachers, and high school test–retest An electronic questionnaire form (appendix A) was emailed to the dean’s offices of 7 departments and distributed to all students in groups at the end of the summer semester, but before final exams. Data collection for the current project was held from June to July 2022 using the Computer Assisted Web Interviewing method. The research concentrated on attitudes towards studying among students who continue their studies at university (BA, MA programs) or were students of Kryvyi Rih State University as of February 24, 2022, regardless of their status at the time of the survey (N = 688). In total, 688 students who participated in the research answered our questionnaire at the end of the year (N = 688; 81.5% female, 28.5% male). The sample included students without stratification by specialties and courses. 4. Results Data were analyzed with IBM SPSS 20. Firstly, we investigated the level of the computer web interview method. The majority of students (87%) rate the disciplines that were taught from February to June 2022 as necessary for their future professional activities (figure 4). This coincides with the all-Ukrainian study of assessing the attitude of students toward higher education [30]. 1.1- 1.4, 1.20 questions (Design and Approval of Programs) keep the same proportions and trends: 1.1. I need discipline for my future professional activity (𝑀𝑀 = 4.5, 𝜎 = 0.95). 1.2. The discipline contains useful material (𝑀𝑀 = 4.6105, 𝜎 = 0.77779). 1.3. The discipline is logically connected with other disciplines (𝑀𝑀 = 4.4390, 𝜎 = 0.93876). 1.4. Discipline contributes to the formation of skills and abilities (𝑀𝑀 = 4.5581, 𝜎 = 0.85284). 1.20. In general, it was interesting for me to master this discipline (𝑀𝑀 = 4.4433, 𝜎 = 1.01645). 357 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 Table 1 Descriptive statistics (N = 688). Question Mean Std. Deviation 1.1 4.503 .952 1.2 4.610 .777 1.3 4.439 .938 1.4 4.558 .852 1.5 4.275 1.066 1.6 4.719 .668 1.7 4.497 .924 1.8 4.590 .829 1.9 4.464 1.015 1.10 4.276 1.101 1.11 4.536 .933 1.12 4.294 1.185 1.13 4.587 .912 1.14 4.513 .981 1.15 4.560 .900 1.16 4.593 .897 1.17 4.724 .725 1.18 4.592 .881 1.19 3.727 1.429 1.20 4.443 1.016 Valid N (listwise) 688 The questions that indicated the style of teaching (Teaching and Assessment): 1.6. The professor is fluent in educational material and modern scientific information (𝑀𝑀 = 4.7195, 𝜎 = 0.66833); 1.7. The professor motivates students to independently search for information in depth (𝑀𝑀 = 4.4971, 𝜎 = 0.92514); 1.8. The professor clearly formulates the goals and the training plan (𝑀𝑀 = 4.5901, 𝜎 = 0.82924); 1.9. The educational material is presented in an accessible and interesting way (𝑀𝑀 = 4.4637, 𝜎 = 1.01595); 1.10. The professor uses the latest interactive teaching methods (𝑀𝑀 = 4.2762, 𝜎 = 1.10149). The possible benefits from Student-centered learning, teaching, and assessment: 1.11. The teacher clearly defines the criteria for assessing students’ knowledge (𝑀𝑀 = 4.2936, 𝜎 = 1.18622). 1.12. I have the desire to continue my studies with this professor (other disciplines, term papers, qualification papers) (𝑀𝑀 = 4.5872, 𝜎 = 0.91308); 1.13. The professor is open and friendly with students (𝑀𝑀 = 4.5131, 𝜎 = 0.98155); 1.14. I have always had the opportunity to turn to the professor for clarification or advice; 1.15. The teacher always fairly, objectively, and transparently assesses the knowledge of students (𝑀𝑀 = 4.5596, 𝜎 = 0.90088); 1.16. The professor is tactful and knows how to establish contact with students (𝑀𝑀 = 4.5596, 𝜎 = 0.89804). The questions that indicated the Learning Resources and Student Support, Certification: 1.5. The discipline is provided with the necessary textbooks and teaching materials (𝑀𝑀 = 4.2747, 𝜎 = 1.06628); 1.17. The professor always conducts classes on time and according to the sched- 358 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 Figure 4: Descriptive statistics (N = 688). ule (𝑀𝑀 = 4.5916, 𝜎 = .88183); 1.18. Distance learning was well organized by the professor (𝑀𝑀 = 4.5916, 𝜎 = 0.88183). Only statement 1.19. But, in our opinion, it would be more correct to teach the discipline personally (𝑀𝑀 = 2.7267, 𝜎 = 1.42989) has a declining trend since the study was conducted during hostilities (table 1). 5. Conclusion An important condition for improving the method of self-assessment and feedback to students regarding the effectiveness of learning a course is establishing the criterion validity of such a method [7]. A criterion that ensures reliable and valid feedback results is considered to be self-assessment by students in the mastery of end-to-end competencies provided by the educational program of the specialty. Such self-evaluation shifts the focus of the criterion of the effectiveness of the educational program and individual disciplines to the student’s general vision of his progress in general and special competencies, in particular those that demonstrate the general orientation of the program to the problems of the educational process and subjects of education, the significance of the results of independent work on the Moodle platform in the individual learning trajectory of students. 5.1. Research question 1 Is multiple types of testing methods (valid, reliable, and fair) for assessing the quality of teaching in order to improve data-driven decision-making and ethics in a crisis situation? Just like the Report “Higher Education through the Eyes of Students”, where students rate higher education very highly. In our study, the averages for all questions ranged from 3.72-4.724 359 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 average. The majority of students (figure 4) reported that they had quite a lot of benefits from subjects in general with 𝑆𝐷 = 4.23. Only 2.2% of students reported that they perceived no benefits in general. For benefits relating to Teaching and Assessment, 65.2% of the students reported these benefits 𝑆𝐷 = 4.31. The results of the scale Learning Resources and Student Support, Certification repeated measures showed that highly (79.3%, 𝑆𝐷 = 4.38). This was also the case for Student-centered learning, teaching, and assessment (55.4%, 𝑆𝐷 = 4.28). Considering the results of the study should analyze the content of the survey. In the condi- tions of hostilities, recommending changes in the behavior of a teacher is a violation of personal boundaries. Therefore, the subsequent objects of monitoring can only be those requirements for the organization of the educational process that are enshrined in the regulatory documentation of each particular university: regulations on the organization of the educational process, job de- scriptions, teachers, regulations on the current monitoring of student progress and intermediate certification, etc. It is advisable to include in the content of the questionnaire only those requirements, the fulfillment of which is likely to be problematic. The questionnaire should also contain general information about the language of teaching the discipline, the percentage of attendance at classes in the academic discipline, and the number of points based on the results of the current progress control. When compiling a questionnaire to assess the qualiity of teaching, it is worth considering the model of Slavin [31], who singles out four important criteria: quality of instructions, relevance, motivation and optimization of time with relevant indicators. In addition, although scientists point to the non-ideal nature of the model, the ambiguity of assigning indicators to one or another criterion for determining the quality of education, we believe that the author of the model defines the core of each criterion quite precisely: quality of Instruction (structure, clarity, comprehensibility, variable forms of teaching, speed, use of media, exercise intensity, scope of material, performance expectations and level of difficulty), appropriateness (difficulties degree, adaptivity, problem sensitivity, individual support and counseling, individualization, support orientation), incentives (teaching content and learning goals, avoiding performance anxiety, arousing interest and curiosity, affirmation and reinforcement, positive social climate), time (time available, used time, content orientation, classroom management). Another well-known model consists of three dimensions, namely: efficiency of class manage- ment, potential of the lesson for cognitive activation and constructive support of the learners [19]. The analysis of the theoretical basis for determining the quality of education, the technology of conducting surveys to assess the quality of education in all its aspects allows us to draw certain conclusions about the structure and content of the questionnaire, which should include questions about the quality of teaching and the organization of the educational process: the quality of instruction, the presentation of the material, its presentation; use of media resources; quality of educational content, its scientific level; time manager; ability to interest, motivate to study the subject. 360 https://doi.org/10.55056/cte.564 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 5.2. Research question 2 What are the ways of anti-crisis digital transformation of teaching at the university in the conditions of hostilities based on decision-making based on survey data? The second modification of the questionnaire gave a 96 test completion rate with a completion time of 12-23 minutes. However, the results of the survey do not give obvious statistical results, which in the conditions of crisis management provide a basis for forming changes in curricula, schedules, number of hours and planning of training retraining for faculty. This encourages further development of the types of testing and data collection based on stratification of the sample according to the disciplines that received the highest marks in groups of students and disciplines that received the lowest marks in groups. This implies a different research design. For further research based on the data, we prospectively assume that with the help of the survey it is likely to: 1) to form a rating of the discipline: relevance in terms of hard skills formation, the novelty of the material, accessibility of educational material, preference for hybrid, synchronous / asynchronous learning, transparency of course evaluation, complexity of the course. 2) to form a component of the teacher rating: student-centered teaching, teaching and support styles. 3) to identify empirically the indicators of student dropout and loss of motivation in the training system. 4) to form promising areas of digital transformation of teaching that support the motivational component of student participation. 6. Data availability statement The datasets generated for this study are available on request to the corresponding authors. 7. Ethics statement Ethical review and approval were not required for the study on human participants in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. References [1] Alekseyev, A.N., Konovalova, N.A., Lozova, C.A. and Trofimrnko, P.E., 2015. Importance and complexity of test tasks at evaluation of activity of university professors. Bulletin of the Kyiv National University of Technologies and Design. Technical Science Series, 2(84), pp.240–246. Available from: https://tinyurl.com/2p8ph7rf. [2] Ang, K.L.M., Ge, F.L. and Seng, K.P., 2020. Big Educational Data & Analytics: Survey, Architecture and Challenges. IEEE Access, 8, pp.116392–116414. Available from: https: //doi.org/10.1109/ACCESS.2020.2994561. 361 https://doi.org/10.55056/cte.564 https://tinyurl.com/2p8ph7rf https://doi.org/10.1109/ACCESS.2020.2994561 https://doi.org/10.1109/ACCESS.2020.2994561 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 [3] Beatty, B.J., 2019. Hybrid-Flexible Course Design: Implementing student-directed hybrid classes. Provo, Utah: EdTech Books. Available from: https://edtechbooks.org/hyflex. [4] Beauducel, A. and Wittmann, W.W., 2005. Simulation Study on Fit Indexes in CFA Based on Data With Slightly Distorted Simple Structure. Structural Equation Model- ing: A Multidisciplinary Journal, 12(1), pp.41–75. Available from: https://doi.org/10.1207/ s15328007sem1201_3. [5] Bloom, B.S., 1994. Reflections on development retrospective. In: L.W. Anderson and L.A. Sosniak, eds. Bloom’s Taxonomy: A Forty-year Retrospective. Chicago, IL: The National Society for the Study of Education, Yearbook of the National Society for the Study of Education, pp.1–8. [6] Bloom, D.E., Canning, D. and Sevilla, J., 2003. The Demographic Dividend: A New Perspective on the Economic Consequences of Population Change. 1st ed. RAND Corporation. Available from: http://www.jstor.org/stable/10.7249/mr1274wfhf-dlpf-rf-unpf [Accessed 2023-03- 11]. [7] Braun, E., Gusy, B., Leidner, B. and Hannover, B., 2008. Das Berliner Evaluationsinstrument für selbsteingeschätzte, studentische Kompetenzen (BEvaKomp). Diagnostica, 54(1), pp.30– 42. Available from: https://doi.org/10.1026/0012-1924.54.1.30. [8] Braun, E. and Leidner, B., 2009. Academic Course Evaluation: Theoretical and Empirical Distinctions Between Self-Rated Gain in Competences and Satisfaction with Teaching Behavior. European Psychologist, 14(4), pp.297–306. Available from: https://doi.org/10. 1027/1016-9040.14.4.297. [9] Braun, V. and Clarke, V., 2006. Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), pp.77–101. Available from: https://doi.org/10.1191/1478088706qp063oa. [10] Cabinet of Ministers of Ukraine, 2022. On approval of the Strategy for the Development of Higher Education in Ukraine for 2022-2032. Available from: https://zakon.rada.gov.ua/ laws/show/286-2022. [11] Chorna, O.V., 2020. Monitoring of quality of higher education in Ukraine and Germany: comparative aspect. Educational Dimension, 2, p.213–225. Available from: https://doi.org/ 10.31812/educdim.v54i2.3869. [12] Dochy, F.J.R.C. and Alexander, P.A., 1995. Mapping prior knowledge: A framework for discussion among researchers. European Journal of Psychology of Education, 10(3), pp.225– 242. Available from: https://doi.org/10.1007/BF03172918. [13] Dübbers, F. and Schmidt-Daffy, M., 2021. Self-determined motivation for data-based decision-making: A relevance intervention in teacher training. Cogent Education, 8(1), p.1956033. Available from: https://doi.org/10.1080/2331186X.2021.1956033. [14] Erpenbeck, J., 2003. Der Programmbereich “Grundlagenforschung”. Zwei Jahre “Lernkul- tur Kompetenzentwicklung”: Inhalte – Ergebnisse – Perspektiven. Berlin, QUEM-Report, vol. 79, pp.7–89. Available from: https://www.abwf.de/content/main/publik/report/2003/ Report-79.pdf. [15] Govermental portal, 2020. Education and Science Reform. Available from: https://www. kmu.gov.ua/en/reformi/rozvitok-lyudskogo-kapitalu/reforma-osviti. [16] Irvine, V., Code, J. and Richards, L., 2013. Realigning Higher Education for the 21st-Century Learner through Multi-Access Learning. MERLOT Journal of Online Learning and Teaching, 9(2), pp.172–185. Available from: https://jolt.merlot.org/vol9no2/irvine_0613.pdf. 362 https://doi.org/10.55056/cte.564 https://edtechbooks.org/hyflex https://doi.org/10.1207/s15328007sem1201_3 https://doi.org/10.1207/s15328007sem1201_3 http://www.jstor.org/stable/10.7249/mr1274wfhf-dlpf-rf-unpf https://doi.org/10.1026/0012-1924.54.1.30 https://doi.org/10.1027/1016-9040.14.4.297 https://doi.org/10.1027/1016-9040.14.4.297 https://doi.org/10.1191/1478088706qp063oa https://zakon.rada.gov.ua/laws/show/286-2022 https://zakon.rada.gov.ua/laws/show/286-2022 https://doi.org/10.31812/educdim.v54i2.3869 https://doi.org/10.31812/educdim.v54i2.3869 https://doi.org/10.1007/BF03172918 https://doi.org/10.1080/2331186X.2021.1956033 https://www.abwf.de/content/main/publik/report/2003/Report-79.pdf https://www.abwf.de/content/main/publik/report/2003/Report-79.pdf https://www.kmu.gov.ua/en/reformi/rozvitok-lyudskogo-kapitalu/reforma-osviti https://www.kmu.gov.ua/en/reformi/rozvitok-lyudskogo-kapitalu/reforma-osviti https://jolt.merlot.org/vol9no2/irvine_0613.pdf CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 [17] Ishutina, O.Y., 2014. E-portfolio by means of Google Apps in the monitoring of lin- hvomethodological competence of students’ philologists. CTE Workshop Proceedings, 2, p.96–99. Available from: https://doi.org/10.55056/cte.193. [18] Kovalchuk, V., Maslich, S. and Movchan, L., 2023. Digitalization of vocational education under crisis conditions. Educational Technology Quarterly. Available from: https://doi.org/ 10.55056/etq.49. [19] Kunter, M. and Voss, T., 2011. Das Modell der Unterrichtsqualität in COACTIV: Eine multikriteriale Analyse. In: M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss and M. Neubrand, eds. Professionelle Kompetenz von Lehrkräften – Ergebnisse des Forschungspro- gramms COACTIV. Münster: Waxmann, pp.85–113. Available from: https://www. researchgate.net/publication/50809729. [20] Machova, R., Komarkova, J. and Lnenicka, M., 2016. Processing of big educational data in the cloud using Apache Hadoop. 2016 International Conference on Information Society (i-Society). pp.46–49. Available from: https://doi.org/10.1109/i-Society.2016.7854170. [21] Marsh, H.W., Hau, K.T. and Wen, Z., 2004. In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler’s (1999) Findings. Structural Equation Modeling: A Multidisciplinary Journal, 11(3), pp.320–341. Available from: https://doi.org/10.1207/ s15328007sem1103_2. [22] Miller, A.N., Sellnow, D.D. and Strawser, M.G., 2021. Pandemic pedagogy challenges and opportunities: instruction communication in remote, HyFlex, and BlendFlex courses. Communication Education, 70(2), pp.202–204. Available from: https://doi.org/10.1080/ 03634523.2020.1857418. [23] National agency for higher education quality assurance, 2019. Strategy of the National agency for higher education quality assurance to 2022. Available from: https://en.naqa. gov.ua/?page_id=642. [24] Piskurska, H.V., 2020. The technology of teaching quality online assessment. Dukhovnist osobystosti: metodolohiia, teoriia i praktyka, (1(94)), pp.192–201. Available from: http: //nbuv.gov.ua/UJRN/domtp_2020_1_19. [25] Polhun, K., Kramarenko, T., Maloivan, M. and Tomilina, A., 2021. Shift from blended learning to distance one during the lockdown period using Moodle: test control of students’ academic achievement and analysis of its results. Journal of Physics: Conference Series, 1840(1), p.012053. Available from: https://doi.org/10.1088/1742-6596/1840/1/012053. [26] Proiekty tsyfrovoi transformatsii, 2021. Available from: https://plan2.diia.gov.ua/projects. [27] Protska, S.M., 2015. Components of computer-oriented methods of formation of pro- fessional competence of future philologists. CTE Workshop Proceedings, 3, p.251–259. Available from: https://doi.org/10.55056/cte.273. [28] Rosa, M.J., Williams, J., Claeys, J., Kane, D., Bruckmann, S., Costa, D. and Rafael, J.A., 2022. Learning analytics and data ethics in performance data management: a benchlearning exercise involving six European universities. Quality in Higher Education, 28(1), pp.65–81. Available from: https://doi.org/10.1080/13538322.2021.1951455. [29] Saukh, P., Nabok, M., Kizilov, O. and Kuzina, I., 2021. The quality of higher education in ukraine through the eyes of students. Herald of the National Academy of Educational Sciences of Ukraine, 3(2), pp.1–8. Available from: https://doi.org/10.37472/2707-305X-2021-3-2-11-5. 363 https://doi.org/10.55056/cte.564 https://doi.org/10.55056/cte.193 https://doi.org/10.55056/etq.49 https://doi.org/10.55056/etq.49 https://www.researchgate.net/publication/50809729 https://www.researchgate.net/publication/50809729 https://doi.org/10.1109/i-Society.2016.7854170 https://doi.org/10.1207/s15328007sem1103_2 https://doi.org/10.1207/s15328007sem1103_2 https://doi.org/10.1080/03634523.2020.1857418 https://doi.org/10.1080/03634523.2020.1857418 https://en.naqa.gov.ua/?page_id=642 https://en.naqa.gov.ua/?page_id=642 http://nbuv.gov.ua/UJRN/domtp_2020_1_19 http://nbuv.gov.ua/UJRN/domtp_2020_1_19 https://doi.org/10.1088/1742-6596/1840/1/012053 https://plan2.diia.gov.ua/projects https://doi.org/10.55056/cte.273 https://doi.org/10.1080/13538322.2021.1951455 https://doi.org/10.37472/2707-305X-2021-3-2-11-5 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 [30] School of Political Analysis of NaUKMA, 2022. Survey “Higher education in the conditions of war through the eyes of students”. Available from: https://naqa.gov.ua/2022/09. [31] Slavin, R.E., 1983. When does cooperative learning increase student achievement? Psycho- logical bulletin, 94(3), p.429. Available from: https://doi.org/10.1037/0033-2909.94.3.429. [32] Staufenbiel, T. and Hartz, C., 2000. Organizational Citizenship Behavior: Entwicklung und erste Validierung eines Meßinstruments [Organizational Citizenship Behavior: De- velopment and validation of a measurement instrument]. Diagnostica, 46(2), pp.73–83. Available from: https://doi.org/10.1026/0012-1924.46.2.73. [33] Sultanova, L.Y., Tsiuniak, O.P., Milto, L.O., Zheludenko, M.O., Lyktei, L.M., Petrenko, L.M. and Uchitel, A.D., 2021. The potential of Google Classroom web service for lecturers of higher educational establishments under pandemic conditions. CTE Workshop Proceedings, 8, p.346–365. Available from: https://doi.org/10.55056/cte.279. [34] Thiel, F., Blüthmann, I. and Watermann, R., 2012. Konstruktion eines Fragebogens zur Erfassung der Lehrkompetenz (LeKo). In: B. Berendt, H.P. Voss and J. Wildt, eds. Neues Handbuch Hochschullehre. Lehren und Lernen effizient gestalten. Berlin: Raabe, vol. I. Evaluation. Veranstaltungsevaluation. [35] Tømte, C.E., Fossland, T., Aamodt, P.O. and Degn, L., 2019. Digitalisation in higher education: mapping institutional approaches for teaching and learning. Quality in Higher Education, 25(1), pp.98–114. Available from: https://doi.org/10.1080/13538322.2019.1603611. [36] Vasyliuk, T.G., 2021. Model of formation of social competence of students in the process of studying the disciplines of the humanitarian cycle. Educational Dimension, 4, p.171–182. Available from: https://doi.org/10.31812/educdim.v56i4.4383. [37] Wentzel, K.R., 1999. Social-motivational processes and interpersonal relationships: Impli- cations for understanding motivation at school. Journal of Educational Psychology, 91(1), p.76–97. Available from: https://doi.org/10.1037/0022-0663.91.1.76. [38] Westermann, R., Spies, K., Heise, E. and Wollburg-Claar, S., 1998. Bewertung von Lehrver- anstaltungen und Studienbedingungen durch Studierende: Theorieorientierte Entwicklung von Fragebögen. Empirische Pädagogik, 12, pp.133–166. A. Appendix 1 Student Course Evaluation “Survey on the quality of teaching disciplines” Instruction “Dear Student / Dear Student! The University Administration invites you to take part in a survey on the level and quality of teaching disciplines. Your answers will help to improve the educational process and improve the quality of education at the university. The survey is conducted anonymously. Thank you in advance for participating in the survey!” 20 questions on a 5-point scale, 3 questions with answer options, and 1 open-ended question. For all psychological variables, respondents gave answers on a 5-point Likert scale from “(1) never true” to “(5) almost always true” in Google Forms. Design of Programs: 1.1. I need discipline for my future professional activity. 364 https://doi.org/10.55056/cte.564 https://naqa.gov.ua/2022/09 https://doi.org/10.1037/0033-2909.94.3.429 https://doi.org/10.1026/0012-1924.46.2.73 https://doi.org/10.55056/cte.279 https://doi.org/10.1080/13538322.2019.1603611 https://doi.org/10.31812/educdim.v56i4.4383 https://doi.org/10.1037/0022-0663.91.1.76 CTE Workshop Proceedings, 2023, Vol. 10: CITEd-2022, pp. 346-365 https://doi.org/10.55056/cte.564 1.2. The discipline contains useful material. 1.3. The discipline is logically connected with other disciplines. 1.4. The discipline contributes to the formation of skills and abilities. 1.20. In general, it was interesting for me to master this discipline. Style of teaching (Teaching and Assessment): 1.6. The professor is fluent in educational material and modern scientific information. 1.7. The professor motivates students to independently search for information in depth. 1.8. The professor clearly formulates the goals and the training plan. 1.9. The educational material is presented in an accessible and interesting way. 1.10. The professor uses the latest interactive teaching methods. Student-centered learning: 1.12. I have the desire to continue studying with this teacher (other disciplines, coursework, qualification work). 1.13. The teacher is open and friendly with students. 1.14. I always had the opportunity to turn to the teacher for clarification or advice. 1.16. The teacher is tactful and knows how to establish contact with students. Learning Resources and Student Support, Certification: 1.5. The discipline is provided with the necessary textbooks and teaching materials. 1.11. The teacher clearly defines the criteria for assessing students’ knowledge. 1.17. The professor always conducts classes on time and according to the schedule. 1.18. Distance learning was well organized by the professor. 1.19. But, in my opinion, it would be more correct to teach the discipline personally. 365 https://doi.org/10.55056/cte.564 1 Background context 2 Objectives 2.1 Educational data-driven decision-making 2.2 Data-driven decision-making in higher education in Ukraine: mechanisms of state regulation 2.3 Data-driven decision-making in university management: research 3 Research methods 3.1 Methods 3.2 Data analysis methods 3.3 Participants 4 Results 5 Conclusion 5.1 Research question 1 5.2 Research question 2 6 Data availability statement 7 Ethics statement A Appendix 1