Orhan, A. (2022). The use of Learning Management System during the Covid-19 Pandemic: How does it work at Bursa Uludağ University? Online Journal of Education and Teaching (IOJET), 10(1). 348-372. Received : 07.09.2022 Revised version received : 28.11.2022 Accepted : 29.11.2022 THE USE OF LEARNING MANAGEMENT SYSTEM DURING THE COVID-19 PANDEMIC: HOW DOES IT WORK AT BURSA ULUDAĞ UNIVERSITY? (Research article) Arzu Orhan 0000-0002-0907-1097 Bursa Uludağ University, Faculty of Education, Department of Foreign Language Education mollaoglu@uludag.edu.tr Correspondence: mollaoglu@uludag.edu.tr Biodata(s): Arzu Orhan has been a lecturer at Bursa Uludağ University / German Language Teaching in Bursa since 1996. The main focus of her work is on language practice. She has participated in various training seminars at the Goethe Institute in Germany and abroad. Her research interests include teaching German to young learners and teaching practice. Copyright © 2014 by International Online Journal of Education and Teaching (IOJET). ISSN: 2148-225X. Material published and so copyrighted may not be published elsewhere without written permission of IOJET. mailto:mollaoglu@uludag.edu.tr http://orcid.org/xxxx International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 349 THE USE OF LEARNING MANAGEMENT SYSTEM DURING THE COVID-19 PANDEMIC: HOW DOES IT WORK AT BURSA ULUDAĞ UNIVERSITY? Arzu Orhan mollaoglu@uludag.edu.tr Abstract During the computer and internet era, there have been a lot of developments in the education field with increasing information technologies. These positive effects have been felt much more for the last few years, especially during the Covid-19 pandemic. This research investigated the use of the Learning Management System-UKEY at Bursa Uludağ University in Turkey by using a survey that includes six questions. In this context, it was aimed to determine the experiences of using UKEY on courses within the transition of lectures to distance education and digital change. The study was carried out on 42 academics from different institutes. It has been observed that academics mostly use the learning platform to share their course contents. As a significant result, it was found that the academics were more reluctant to encourage their students in online discussions. In doing so, it can be said that one of the main obstacles to using interactive online tools for supporting student activities is academic attitudes to LMS. However, it was comprehended that if LMS was used appropriately, it could positively affect students’ motivation and contribute precisely to the education process. According to the results, it has been seen that the academic approaches, which have an essential role in the transformation of technology, were vitally important during the change period and affected the success of the digital/technology transformation. The results revealed that it was essential to make face-to-face learning and increase the efficiency of the learning platforms. Consequently, it can be emphasized that learning with interactive tools on learning platforms will be planned more shortly within the scope of LMSs, which is considered an alternative solution to face-to-face training. Keywords: Higher Education, Learning Management Systems, UKEY, Covid-19 pandemic 1. Introduction There have been many developments in every area, even in the education field, regarding the everyday usage of information technologies for the last decades. The internet, which has become a necessity of the Information period (computer period, digital period, new media period, or electronic period), has been an indispensable part of educational technology. With the development of telecommunication and computer technologies, especially with the spread of the internet, the most significant change has been in communication. Organizations will be one level forward in the competitive conditions if they can monitor changes and use information technologies like the internet and Learning Management Systems (LMSs) prominently in their strategies. LMS is one of the fast-developing technologies broadly used in schools and higher education, and the increasing use of LMS forms a new prototype known as e-learning. Consequently, universities worldwide started reconsidering their policies according to modern technologies that help reach their purpose (Naidu, 2006; Alharbi & Drew, 2014). Organizations such as private and governmental companies must prepare their employees for ongoing developments if they don’t want to miss the Information Age. This situation can also apply to all school types. In this age, rapid developments in communication technologies mailto:mollaoglu@uludag.edu.tr Orhan 350 affect the structure and form of education, forcing educators to develop new education programs and learning-teaching models. It has been reported on the rapid acquisition of knowledge and the rapid adaptation of teachers and learners to rapid changes in learning platforms in LMSs (Meyer, 2016). Thus, learning platforms are used at schools and universities, especially for digital educational contexts. They enable the transfer and organization of material, often offer various exchange channels, and allow learning progress to be technically tracked. All these functions must be decided on whether they are the helpful case-by-case basis. According to Meyer (2016), LMSs are integral to schools and universities. They see themselves as web-based “Software systems for organization, control, and communication for learning and teaching” (Kultusministerkonferenz, 2016) and enable learning content creation, modification, and publication. LMSs are mainly used “where many users encounter a sustained need for learning” (Müller, 2018). As web-based platforms, they address various educational tasks, from administering users and courses to assigning roles and rights to multiple options for communication and cooperation between teachers and learners. In addition, in the context of digitization in schools, universities, and companies, LMSs are considered decisive markers for the change and reorientation of teaching (Borchert, Fritzenberg et al., 2017). These observed changes in Information and Communication Technologies (ICT) are broadly used in education and other industry, business, and government areas. However, the usage of information technologies has gained new dimensions daily and continues to influence life considerably (Lewis et al., 2005). For example, LMS in education processes would be more critical to get efficient activities during and after the Covid-19 pandemic considering blended training methods adopting both traditional and distance education methods in all aspects of education (Kurnaz & Serçemeli 2020, Aydemir, 2021). Due to the Covid-19 pandemic, all educational institutions, including universities, were forced to switch to digital teaching and learning. Previous teaching and learning formats had to be implemented exclusively online in the short term, which presented many universities, lecturers, and learners with major or minor hurdles (Kerres, 2020). Instead of a well-thought-out and planned digital transformation, students and lecturers were quickly confronted with many innovations and requirements because of the Covid-19 pandemic. During this time, they had to switch to e-learning. Kerres (2018) defines E-learning as providing and using teaching and learning material using electronic media. It is a term for all forms of media-supported learning that integrate multimedia and communicative technologies (Kerres, 2018). It is considered a significant and essential key for innovative university teaching (Persike & Friedrich 2016, Schünemann & Budde 2018, Metzner et al., 2019). All these studies have displayed that universities increasingly rely on digital formats for teaching and learning in training and further education. However, various organizational factors must be considered when designing the learning conditions and the learning process. According to Kreidl (2011), learning with digital media brings many new challenges, so universities should establish supportive measures on content and technical level (Kreidl, 2011). The organizational design also includes the technical implementation of the offer. Especially, user-friendliness and functionality of digital learning environments shape study behavior (Persike & Friedrich, 2016). For example, students demand transparency about the examinations, structured processes, and clear communication. Moreover, Ehlers (2011: 180) emphasizes that “the learning platform - in the sense of technical functionalities - is a factor that can influence motivation.” (Ehlers, 2011). Digital learning opportunities have been developed, tested, and used at many Turkish universities recently. LMS at Bursa Uludağ University (BUÜ), which calls “UKEY,” is one of them. From this point of view, it is thought that this study can determine the current situation in the context of distance International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 351 learning and distance education in light of scientific data and provide factual data to institutions and researchers so that plans can be made in line with attainable targets. 2. Review of literature 2.1. Learning Management Systems (LMS) In prior research, it can be seen that the applications of computers to education are dated back 1950’s and are named differently, such as computer-based order (CBI), computer-assisted order (CAI), and computer-assisted learning (CAL). In the beginning, LMS had some other expression, integrated learning system (ILS), which provides functionality beyond instructional content, such as managing and tracking individual directions and incorporating crosswise the system (Bailey, 1992; Becker, 1992; Brush et al., 1999; Parr & Fung, 2001; Szabo, 2002; Watson & Watson, 2007). These studies explain LMS as a Learning Management System initially used to describe the management system components, content-free and separate from the courseware. According to Watson & Watson (2007), the expression LMS currently explains several educational computer applications. Other studies also reported that LMS is the frame that comprises all ingredients of the learning procedure. An LMS is the basic structure that presents and handles educational content, realizes, and evaluates personal and organizational learning or teaching purpose, follows the progression toward meeting those aims, and collects and delivers data to observe the learning procedure of an organization as a whole (Szabo, 2002). Furthermore, an LMS provides information and manages course application and management, skills gap analysis, tracking, and reporting (Gilhooly, 2001). Universities in the USA began to combine the programs used in the education process with the internet in the early nineties, thus creating the first LMSs (Newber et al., 1994) applications. Since that time, most LMS have tools for Online Asynchronous Discussion (OAD) planned to encourage learner(s)-to-learner(s) interaction (Murphy, 2004; Murphy & Loveless, 2005). Many researchers are optimistic about using LMSs to create effective learning environments and improve the learning experience by supporting learner-centered techniques and collaborative methods (Ubell, 2000; Wilson & Stacey, 2004; Blin & Munro, 2008; Ladyshewsky & Gardner, 2008; Bush & Mott, 2009; Fındık & Özkan, 2010). There are studies about discussion forums and other tools that may facilitate group homework and enhance the learning experience through interaction and sharing of information. They can also elicit group identity and improve learning by promoting a community of learners (Hopperton, 1998; Murphy 2004; Irwin & Berge, 2006). It has also been argued that, even if courses are given on- campus, OADs elicit students to structure and organize their thoughts better than chatting in real-time or face-to-face discussions (Cheng et al.; 2011). Nowadays, it is observed that computer-based learning platforms are increasing in educational institutions. Moving the education environment partially or entirely to the internet environment has gained importance in using software and hardware technologies in education. Institutions now need integrated educational software that complies with their academic standards, curricula, and assessment tools. In previous researches, some of which have been above-mentioned, it can be read that computer usage in education started in the 1950s (Watson & Watson, 2007). Since then, many terminologies and concepts forced to use computers in education have entered. LMSs are one of them. They are briefly defined as interactive tools that enable learning and teaching processes to be carried out over the internet with various tools and features it contains. It is an integrated system that allows the management of educational content, monitoring of learners and lecturers, and allows individualizing the learning and Orhan 352 teaching processes. The primary difference between LMSs and other computer terms is that LMSs systematically cover the entire learning and teaching procedure. When we look at the definitions of the term LMS in different sources, it is seen that Network Management is defined as a software package that provides the management, distribution, and delivery of learning resources to students (Dictionary, 2022). Many LMSs enable access and manage learning content from anywhere and at any time. In Wikipedia, the LMS is expressed as management software that allows students to choose courses, record course, content presentation, measurement and evaluation procedures, and user information in distance or blended education (Wikipedia, 2022). Besides, LMSs can be defined as software that offers different options for learning and managing these experiences, as well as platforms that provide the management of teaching and learning processes, access information for educational purposes, and collect information sharing and communication processes. In line with this information, LMS can be explained as software that provides management of learning activities. Services in LMS can differ from one system to others. However, basic services may contain admission control, performance administration, communication skills, evaluations, study timetable documentation, and supply of learning content (Cavus & Ala’a, 2009). It is also seen that these systems offer functions like presenting the learning material, sharing and talking about the learning topics, organizing courses, carrying out homework’s and exams, providing feedback about this homework and exams, managing the learning materials, and also giving a chance to make notes about students, lecturers, and the system (Paulsen, 2002). In this case, an LMS can enable a meaningful division of activities in the course process, simple communication between learners, active participation, and feedback to institutions and learners. In this respect, LMSs, which provide an infrastructure with advanced tools and features, are frequently used in such educational environments (Rubin et al., 2010). Also, these systems are mainly used for purposes suchlike as distribution and management of content, as well as providing opportunities for interaction between learners and lecturers (Aydın, 2003; West et al.; 2007). Furthermore, to make course materials more accessible on the web, LMSs can also present lessons with more exciting and extensive content, exchange information in interaction with students and lecturers, and promote learning continuity (Dutton et al., 2004). Concerning studies and books on the usage of LMSs from the beginning of this century, it becomes evident that researchers support the creation of student-centered methods and effective learning environments. Accordingly, researchers stated that LMSs had improved the learning experience, and these developments continue (Ubell, 2000; Blin & Munro, 2008; Bush & Mott, 2009; Fındık & Özkan, 2010). Related to this, many researchers have claimed that LMSs can improve learning (Hopperton, 1998; Murphy, 2004; Irwin & Berge, 2006). It is also stated that the use of other computer-based learning methods to provide online discussions and interaction in the learning process within an LMS can significantly improve the learning experience (Yarusso, 1992; Anderson & Kanuka, 1998; Cronjé, 2006). Studies were conducted to determine LMSs and their usage areas. In their research, Malikowski, Thompson, and Theis (2007) investigated which lecturers adopted the LMS in the learning process and divided the interactions into three groups according to their frequency of use. Accordingly, the most common aim of this study was to convey the content to the learners. In this context, the first group was focused on uploading files to the learning environment, transferring the learning content to the system, entering announcements, and following the learner’s success. The second group represented the assessment of learners and the interaction in the class. This group was also assigned features such as interactive tools and online discussions, messaging, classroom interaction, quiz assessments, and homework submissions. International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 353 Finally, the rarely used category included the course evaluation. For example, surveys measure course satisfaction and lecturers’ performance (Malikowski et al., 2007). In another research, Lonn and Teasley (2009) discussed the benefit to the learners and lecturers and the log records obtained from the system related to Sakai, an online learning platform included in the LMS and used to promote face-to-face learning. While the lecturers found it beneficial to provide the learner-lecturer and learner-learner interactions in this study, the learners considered the LMS to save time. The Log Management System data of this research showed that activities with low interaction rates, such as content sharing, assignments, and announcements, are about 95% of the use of the LMS and 5% of the speed of interaction with higher interaction tools such as instant messaging, discussion environment (Lonn & Teasley, 2009). The research conducted by Herse and Lee (2005) showed that students are most satisfied with sharing the course materials in the LMS environment (Herse & Lee, 2005). It turns out that the function of LMSs is to facilitate learning activities and perform them more systematically and planning. The learning method is constantly improved if learning activities are evaluated through these systems. Since the student’s actions are monitored, the learners are helped when necessary (Duran et al., 2006). However, educational environments need to be carefully planned to help learners organize their learning lives. A successful learning experience matches the learning environment and the learner’s needs (Alkan, 1987, Federico, 2000). In well-structured interactive environments where learners have access to different sources and other people, learners, according to (Aydın, 2003), can simultaneously and asynchronously with lecturers and actively interact with content, thus achieving a meaningful and lasting learning process. Also, one can design and provide more hands-on classroom activities and materials as long as one tries to understand better learner behavior and preferences in LMS (Mogus et al., 2012). Over the last decade, universities have faced socio-economic and technological changes (Fox, 2007). Therefore, the demand for more flexible courses and study opportunities for students, the similarity of European higher education programs in connection to the Bologna Process, and the pressure to use information and communication technologies can be shown. When evaluated from this point of view, Instructional Management Systems will inevitably occur in most universities worldwide. Studies showed that more than 95% of all replying universities in the USA have accepted one or more LMS (Arroway et al., 2010) and that the same tendency is also in the UK (Browne et al., 2006). 2.2. Using ICT in Higher Education in Turkey Various projects and initiatives have been on the agenda since the mid-1980s. Still, the earliest significant step for information and communication technologies in higher education programs in Turkey was the establishment of the National Academic Network and Information Center (ULAKBIM) in 1996, which is a part of the Turkey Scientific and Technological Research Council (TUBITAK) (Çağlayan & Bener, 2006; Tosun, 2008; Ateş, 2020). The “Strategic Plan of the Ministry of National Education (MoNE), 2015-2019” aimed to improve schools’ ICT infrastructure, support the production of digital content and develop teacher training on how to use these infrastructures and content effectively (Akdur, 2017). As in many countries, the using ICT in education is seen as a strategic matter in Turkey. The premise is that with the development of telecommunication and computer technologies, especially with the spread of the internet, universities have to monitor changes and use information technologies prominently in their practice to keep up the quality of education and prepare students for the future. Turkish government supports ICT development in higher education through national policies and budgets for establishing these systems (Tecim & Orhan 354 Gökşen, 2009). Many plans and procedures intended to encourage the usage of digital instruments were published through the websites of higher education organizations. All institutions have admission to the national network that offers many features of LMS, although some have chosen different systems. (Tosun, 2008). 2.2. LMS (UKEY) at BUÜ BUÜ is a state university established in 1975 in Bursa, Turkey. Besides the central campus, there are seventeen other campuses in distributed locations with thirty-four colleges. The first faculty was the Faculty of Medicine, founded in 1970 within a relationship with the University of Istanbul. In 1974 the Faculty of Economics and Social Sciences at the University of Bursa was founded. The information about the BUÜ demographics is given in Table 1. BUÜ has four institutes, fifteen faculties, three colleges, fifteen vocational schools, and one conservatory. BUÜ has approximately 2,577 academic staff and 65,834 enrolled students. Today, BUÜ is the biggest university in Turkey in the number of students. It also offers courses and programs in most academic disciplines at the master's and doctorate levels (www.uludag.edu.tr). Regarding Turkey's national strategic plan, BUÜ pays attention to including LMS in learning and teaching programs and practices. These attempts aim to promote both academics and students with technical support. Furthermore, workshops and webinars have been conducted to increase academics’ awareness of LMS. However, face-to-face lessons and teaching are still the official training method at the university. The Learning Management Program called “UKEY” (BUÜ Institutional, Educational, and Research Management System) has been prepared to manage, record, monitor, and evaluate all activities carried out in the quality of education and research at BUÜ. UKEY, whose infrastructure is based on the software developed with the facilities at the IT Department, was implemented with the first application launched in the Faculty of Education in the 2013-2014 fall semester. It provides lessons, in- service training, consultation, meetings, seminars, and various educational and managerial activities. This program, which is put into service, works in integration with the “Information Package” and “Student Automation System” created as a result of the studies within the scope of the Bologna Process and makes many data evaluable for users by taking these data from these programs (Alyaz et al., 2014). Table 1. BUÜ demographics Campuses and Faculties Total campus Institute Faculty College Vocation Conservatory 17 4 15 3 15 1 University Member Statistics Professors Associate Professors Assistant Professors Lecturers Instructors Total 652 317 224 595 789 2.577 International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 355 Figure 1. Some screen pictures from UKEY As seen in some screen pictures from UKEY in Figure 1, all of the homework, projects, exams, and other education and training activities prepared by our undergraduate and graduate students on paper can be carried to the computer environment. This program enables students to develop teamwork and individual development by preparing homework and group projects. Forum and visual communication platforms have also been created to eliminate the disruptions in student-academic meetings and communication, mainly due to the increasing workload and lessons. Any task assigned through UKEY is added to the automatically created plan for the user, and the task is informed via e-mail to the person who gives or undertakes the task, and by providing a warning in case the given job is not completed; it also contributes to the planning of the business and educational lives of the users as a job tracking program (www.uludag.edu.tr). 3. Aim and Purpose The literature review reveals that studies on the use LMSs are well documented. It is also well-acknowledged that LMSs used in different organizations, like universities. And academic and student satisfaction with using an LMS is related to their readiness for it. Previous studies mainly focus on measuring the behavior of academics toward LMS systems before the Covid- 19 pandemic period. In response to this gap in the literature, this study aims to investigate the use of UKEY in BUÜ and identify the differences in the use pattern and opinions of lecturers about UKEY. It includes the extent to which digital teaching and learning formats academics used in their classes in 2019-2020 during the Covid-19 pandemic. The results are analyzed to suggest broader utilization of UKEY in BUÜ. The concern of this study resulted from several approaches. First, previous research has not investigated faculty staff behavioral aim to use LMS at BUÜ during the Covid pandemic. Second, the findings of this study will present different inputs and insights into the academics’ perceptions of LMS of other faculties. Third, this study will facilitate future research on LMS within the university context in Turkey. This study selected and revised a survey to adapt the existing LMS acceptance context that may be reprocessed in new academic researches. 4. Methodology It was used an online survey for data collection in this study. Online surveys offer several benefits, such as saving time and costs by overcoming geographic distance. The advantage of a survey is the assurance of anonymity for the participants. On the other hand, a survey enables them to give more thoughtful and honest answers because there are no possible interviewer mistakes. There is no pressure from an interviewer or a group present (Schnell et al., 1999, Orhan 356 Wright, 2005; Alharbi & Drew, 2014). Finally, the online survey used in this study was developed to verify the relationship between questions. The online survey was adapted from the original measurement scales and other literature with some modifications and the required wording changes to fit the context of LMS use (Herse & Lee, 2005; Wright, 2005; Garrote & Pettersson, 2011; Garrote Jurado et al., 2019). To avoid problems that can appear in wording, measurements, and uncertainties, the survey was pre- tested by two volunteers. It was highlighted that such a pre-test is important because wording problems considerably influence the survey’s exactness (Bougie & Sekaran, 2019). 4.1. Research Design and Data Collection Tools The survey consists of two parts. The first part includes a nominal scale to identify participants’ socio-demographic information such as gender, age, academic rank, academic field, and teaching experience (Mitchell, 1998). Six questions are measured according to the study in the second part, as discussed in the survey below. A practical application of LMS should contemplate academics' views using such systems for teaching. With this research, the survey determined the academics’ opinions about the use of UKEY in BUÜ. Thus, the views of academics who use the UKEY platform were presented. The quantitative method was used to evaluate the data in the research. Therefore, the primary aim of the present study is to determine based on the opinions of the academics by using a quantitative design to investigate the following research questions: 1. Do your students use the LMS to collaborate on some projects in your courses? 2. Do you use the LMS to provide your students with study material? 3. Do your students use the LMS to discuss the subject of their courses? 4. Do you think your students discuss the subject of their courses using some other program (such as Facebook or Twitter)? 5. Do you encourage your students to engage in online discussions? 6. Do you think all students should engage in online discussions? The survey was adapted from Garrote’s (2019) study “European Lecturers’ Perceptions of Interactive Tools in Learning Management Systems” (Garrote Jurado et al., 2019). Organized as a four-digit Likert-type survey for three questions, a value of 1-4 of the answers could be selected, choosing “always” (1), “sometimes” (2), “once/very rare” (3), and “never” (4), and in the survey, which was organized as a three-digit Likert type for two questions, the answers were asked to select a value between 1-3 and indicate “always” (1), “sometimes” (2), and “never” (3). In the last question of the survey, it was asked to mark one of the opportunities, which are “Yes, during in most courses.”, “Yes, but only in certain courses.”, “Yes, but voluntarily.”, and “No, those who wish can always discuss if they want.”. Besides, if the academics had any comments they wished to add to each question, they were allowed to write this. This survey was uploaded to the internet and was made with Google Documents to collect the data. It was made available in the academic year 2019-2020 during the Covid-19 pandemic. The survey link was sent to 148 academics’ e-mails. Ten days later, once more, a follow-up e- mail was sent. These two e-mails formed all of the responses for the study. They activated their connections, answered every question, and registered their answers. The academics were informed about the study's purpose, the survey's content, and ethical considerations. After consenting to join the study, the academics could complete the survey. The answers were International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 357 caught up through a Google account and downloaded in Excel as a table to the computer. The results are analyzed to suggest a wider usage of UKEY in BUÜ. Ethical clearance was provided for all participants before the survey. Participation in this survey was voluntary, and data was collected anonymously. This study did not include personal information about the participants. Before starting the survey, all participants were informed about the research's concern and how data was collected. It was clarified that the participation is established on the subjects’ interest, that they are not compulsory to participate, and that they may refuse to participate at any time. Their right to withdraw at any time during the survey was clearly stated. Moreover, (Alharbi & Drew, 2014) recommend that research data in researches should be assured confidently, which was also done in this research. Academics were asked to fill out an online survey about the usage behavior of their digital advertising tool and learning platform (UKEY). During the Covid-19 pandemic, academics were mainly conducted via UKEY, with Google Meet as the primary video conferencing system. The existing LMS (UKEY) was used for the online exams, tasks, and submissions. Online functions on the digital platform (UKEY) also measured the students’ course performance. 4.2. Sampling Technique and Participants Sampling is an approach to making a conclusion based on a small presentation of a specific population when getting feedback from an entire people is hard. (Jemain et al., 2007). The sample in this survey is considered a subset of the chosen academics from different institutes at BUÜ. It is seen that convenience sampling was used in many studies investigating technology acceptance. Therefore, convenience sampling was selected since it was the optimal technique for this study. Finally, the present study used non-probability convenience sampling as a sampling technique. Additionally, this technique quickly provides a better feedback rate (Alharbi & Drew, 2014). The participants were chosen using a practical random sampling method. The information about academics’ contacts and email addresses was obtained from different faculties or departments. The participants in this study were 42 academics working at BUÜ during the 2019-2020 academic year, and they voluntarily participated in the online survey. In context with the survey, participants were given short information about the research goal and its contribution to the subject. 4.3. Data Analysis Academics answered each question and recorded their answers by activating the link sent via Google Documents. Accordingly, the responses were obtained through a Google account and downloaded to the computer. The data were tabulated with Microsoft Excel and Word support and analyzed using Statistical Package for Social Science software (SPSS-IBM) for Windows (version 25). Descriptive statistics were used to describe the survey. The internal consistency of the question scales was examined with Cronbach’s alpha (α) coefficient (CAC) (Cronbach, 1984; Eisinga et al., 2013). CAC can be written as a function of the number of test questions and the average inter-correlation among them as follows in Eq 1: 𝛼 = 𝑁 .𝑐̅ �̅�+(𝑁−1).𝑐̅ Equation (1) Orhan 358 Here, N is equal to the number of questions, 𝑐̅ is the average inter-question covariance among the questions and �̅� equals the average variance. The level of significance adopted for the statistical tests was 5%, that is, P < 0.05. 5. Findings 5.1. Demographic and Descriptive Statistics The study sample in the present research consisted of academics working in different institutes at a state university in Turkey during the 2019-2020 academic year. This study aimed to examine the use of UKEY in BUÜ. Therefore, it has been prepared a survey and then it was analyzed academics’ opinions about UKEY. As mentioned earlier, within the survey described overhead, all participants had access to the demographic section, which is presented first. As presented in Figure 2, of the 148 enrolled academics from 4 institutes invited to participate in the study, 42 respondents completed the online survey. The return rate in the overall response was 28.4%. Although this ratio was slightly low when compared to other research (Alharbi & Drew, 2014), it does not pose a problem-scientifical- theoretical in terms of this study (Smith, 2008). The majority of participants were females in terms of gender, with 28 (66.7%) females and 14 (33.3%). In other words, female academics have contributed much more to the survey. Moreover, academics belonged to different age groups, but the great majority were between 30 and 50 years, with 14.3% from 25 to 30, 40.5% from 31 to 40, 28.6% from 41 to 50, and 16.7% above 50, an age range of 25 to 55 (M=40.0). The results generally reveal that 66.7% of respondents were Assistant Professors of higher ranks. The majority of academics have been teaching at Educational, and Natural Sciences, 45.2 and 28.6%, respectively, and a small part of academics has been working at BUÜ for less than five years (4.8% for 1-3 years and 14.3% for 3-5 years). All demographic results in Figure 2 indicated that the academic persons had a good experience expressing their valid opinions while answering the survey. Figure 2. Some demographic and descriptive information International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 359 5.2. Validity and Reliability Statistics It frequently uses multiple-item scales to quantify constructs that are not directly measurable. Likert-type scales are generally used to describe information such as attitudes, emotions, opinions, personalities, and descriptions, which are gathered in the social sciences (Likert, 1931). In this study, the survey questions were prepared using Likert-type scales to determine the Lecturers’ opinions about UKEY. When using Likert-type scales in the research, it is necessary to calculate and report CAC for internal consistency reliability for scales or subscales (Cronbach 1951). Reliability cares about internal consistency between multiple measurements of variables, and CAC is commonly used to measure it (Hair et al., 2014). The reliability of the survey means that answers are characterized by repetitiveness and are not connected with measurement errors. The evaluation of survey reliability-internal consistency is possible by CAC, which is considered the most important reliability index and is based on the number of questions of the survey, as well as on the correlations between them (Nunnally, 1978; Cronbach, 1984; Anastasiadou, 2010; Anastasiadou, 2011). Responses were exported from the Google online survey hosting platform to analyze the raw data, tabulated with Microsoft Excel and Word, and analyzed using SPSS. The survey data was first evaluated through a reliability analysis for the consistency of the collected scale results. And the reliability analysis was performed as calculated by CAC, which usually ranges between 0 and 1. The closer CAC is to 1.0, the greater the internal consistency of the items in the scale, as per many studies (Nunnally, 1978; Santos, 1999; Gliem & Gliem, 2003, Sekaran & Bougie, 2016; Elsharif, 2019), surveys are examined to have internal consistency reliability when CAC exceeds 0.60 to 0.70, which means that the scale is acceptable. It was used 0.60 as the cutoff (minimum) level for CAC, and the values in each cell were averaged over six questions. Calculated CAC verified the stability of the survey for each set of survey questions in Table 2. All measurements in this study showed a satisfactory reliability level, ranging from 0.590 to 0.700. Generally, the larger the number of items in a scale (N), the more reliable the scale (Nunnally 1978). Hence, if the survey's total number of questions (N) has increased when other factors are constant in this study, CAC can be higher, as seen in Eq 1. Table 2. Questions-Total Statistics Scale Mean if Question Deleted Scale Variance if Question Deleted Corrected Question Total Correlation Squared Multiple Correlation Cronbach’s Alpha if Question Deleted Q2 11.74 9.564 0.311 0.167 0.692 Q3 13.21 9.733 0.387 0.285 0.673 Q4 11.52 7.329 0.606 0.531 0.590 Q5 13.17 9.411 0.460 0.263 0.655 Q6 11.36 7.894 0.534 0.418 0.620 Q7 11.74 8.491 0.331 0.214 0.700 The reliability statistics were calculated, and the results are displayed in Table 3. CAC value was statistically significant and equaled 0.698 for the total number of questions, suggesting that the questions had moderate-satisfactory internal consistency and were considered acceptable in most social science research situations (Bland & Altman, 1997; Anastasiadou, 2011). It can be interpreted as quite reliable since the CAC value is 0.60≤α≥0.80. This value did meet the requirements for satisfactory reliability, as stated by Bland and Altman (Bland & Orhan 360 Altman, 1997). This may be due to the fact that all of the questions had tetrachotomous response categories, and it was slightly low due to the formative scaling (Leon et al., 1995). As a result, the survey can be considered reliable in terms of internal consistency in determining Lecturers’ opinions about UKEY. Table 3. Reliability Statistics Cronbach’s Alpha Cronbach’s Alpha Based on Standardized Questions N of Questions 0.698 0.704 6 5.3. Lecturers’ Opinions about UKEY Today, LMS is the most widely used at universities for teaching activities. Private companies have already practiced LMS routinely for their employee training, and now LMS is more common in many areas, mainly because of the Covid-19 pandemic. In the context of digitization in schools, universities, and companies, LMSs are considered a powerful feature for the change and reorientation of teaching (Wendeborn et al., 2018). Findings obtained from the survey were analyzed in six sections to determine lecturers’ perceptions of LMS at the BUÜ. This section is about UKEY used at university, and it is about the results of lecturers to assess their perceptions of the learning platform. The opinions of lecturers on the use of UKEY differ in some questions. The following section presents the analyzed data and the explained findings. The comments that lecturers would like to add to each question were also evaluated. Table 4. Do your students use UKEY to collaborate on their homework’s and projects in your courses? Answers f % Always 1 2.4 Sometimes 16 38.1 Once/Very rare 15 35.7 Never 10 23.8 The survey answers to the question “Do your students use the LMS to collaborate on their homework and projects in your courses?” was presented in Table 4. Lecturers expressed that nearly half of the students used the LMS for collaborating, with 2.4% from always and 38.1% from sometimes. When using the LMS, each lecturer uses several content elements differently. In the study by Wendeborn, Schneider, and Karapanos (2018), which researches their LMS at their university, forums are used most frequently, followed by working material in the courses. Similar to this finding, it can be said that the results at the BUÜ are nearly the same (Wendeborn et al., 2018). International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 361 Table 5. Do you use UKEY to provide your students with study material? Answers f % Always 32 76.2 Sometimes 7 16.7 Once/Very rare 2 4.8 Never 1 2.4 Considering the findings in Table 5 about the question “Do you use the LMS to provide your students with study material?” the majority of lecturers answered “always,” with 76.2% as the highest value. Besides, some lecturers reported in their comments about their LMS that they think, “This function in UKEY I find most useful” or “And, it provides great convenience” (Translations by the authors). Table 6. Do your students use UKEY to discuss the subject of their courses? Answers f % Always 1 2.4 Sometimes 19 45.2 Once/Very rare -- --- Never 22 52.4 The lecturers surveyed have perceived the transition to digital platforms as labor-intensive and challenging. During a digital changeover, communicative skills are mainly required for digital teaching (Goetz, 2020). While the possibility of immediate feedback was assessed as positive, it turned out that the lectures at BUÜ did not provide the LMS as a choice to discuss the course subject. The findings in Table 6 are related to the question, “Do your students use the LMS to discuss the topics covered in the lessons?”. The lecturers’ answers have revealed that students are not mainly used to discussing the topics covered in the courses’ online discussion environments in LMSs. Nearly half of the students never used the LMS for discussion, with 52.4%. For this question, a lecturer commented, “No, I try to spare as much time as possible for my students in the lesson. I try to make my lessons as interactive as possible” and “Our students prefer to meet face-to-face because they can easily access their teachers rather than in the LMS UKEY.” (Translations by the authors). These comments can be seen as an explanation of why they never use their LMS. Table 7. Do you think your students discuss the subject of their courses, using some other program (such as Facebook or Twitter)? Answers f % Orhan 362 Always 31 73.8 Sometimes 6 14.3 Once/Very rare -- --- Never 5 11.9 The findings in Table 7 belong to the question, “Do you think your students discuss the subject of their courses using some other program (such as Facebook or Twitter)?”. Looking at the answers, it can be seen that most students used a different program than LMS for discussing, with 73.8% from always and 14.3% from sometimes as the highest value. The situation can explain the high value that today’s university students can be seen as digital natives who have no idea what it would be like to live without social media and the internet. Therefore these students are believed to be ready to use the LMS (Hao, 2016). Table 8. Do you encourage your students to engage in online discussions at UKEY? Answers f % Always 4 9.5 Sometimes 5 11.9 Once/Very rare 12 28.6 Never 21 50.0 As is well known, an LMS has various content elements available to users to create online lectures. According to Wendeborn, Schneider, and Karapanos’s study (2018), these content elements are intruded into the course. However, the integration into a lecture says nothing about the actual usage behavior. They have found that access to forums and online discussions is relatively shallow (Wendeborn et al., 2018). A similar situation can also be found in the statements of the lecturers at the BUÜ. In the question in Table 8 on encouraging students to participate in online discussions in LMS, it can be seen that the minority of lecturers gave positive feedback on lecturers’ encouragement of their students, with 9.5% from always and 11.9% from sometimes. This value is found as the lowest value in the survey. From this point of view, it is figured out that lecturers do not encourage their students to participate in online discussions. International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 363 Table 9. Do you think all students should engage in online discussions? Answers f % Yes, during most courses 7 16.7 Yes, but only in certain courses 9 21.4 Yes, but voluntarily 11 26.2 No, those who wish can always discuss if they want 15 35.7 Undoubtedly, the direct, close exchange between students and their lecturers should be regarded as the most important factor. Wendeborn, Schneider, and Karapanos’s study (2018) found that students perceived direct, personal conversation and knowledge transfer in face-to- face lessons as more effective and demanded direct communication. In connection with this finding, the lecturers also stated that the students should decide when to participate in online discussions. The results in Table 9 engage in the question, “Do you think that all students should engage in online discussions?”. Considering the answers, lecturers responded, “No, students can always discuss if they want.” with 35.7%. It is understood that lectures do not encourage and engage their students in online discussions. 6. Discussion The present study investigated lecturers’ opinions on using their LMS (UKEY) at BUÜ. According to Table 4, nearly half (40.5%) of the students used the LMS for collaborating. The results demonstrated that only some content elements are integrated into the LMS's online courses. Also, some content is only used by a small proportion of users. This is consistent with what Wendeborn, Schneider, and Karapanos (2018) found in their studies (Wendeborn et al., 2018). If the range of functions were reduced, hardly anyone would miss anything. Considering the software components, such as the ability of lecturers and learners to manage content, interactions, and reporting in LMSs, it was found that the lecturers at BUÜ use most of their LMS to provide course material to students. Similar results are in the findings reported by Garrote and Pettersson (2007) and Garrote and Pettersson (2011) (Garrote & Pettersson, 2007; Garrote & Pettersson, 2011). Their results showed that lecturers mostly use LMSs to provide access to course materials to their students. Only very few students used the online interaction function regularly. It was seen in Table 5 that the majority (76.2%) of lecturers at BUÜ use UKEY to spread the course content and retrieve documents for access. In this function, it can be seen that UKEY has supported the students primarily by providing textbooks, texts, or multimedia tools. Furthermore, it has facilitated the flow of information between the lecturer and the students. In addition, LMS enables teachers to be supported in various ways. For example, materials can be linked to online use of student support and a calendar so that certain documents can only be viewed at a specific time, for example, when an exam is passed. Another feature is that it allows online interaction. In this case, many learning platforms allow many modern LMS users to interact and collaborate. Looking at Table 6, the results showed that nearly half (52.4%) of the students never used the LMS to discuss the topics covered in the class. On the one hand, this situation can be explained by technical problems, such as the fact that the students do not have the internet and have difficulties participating in the discussion. Second, the results in this study support the findings in Perry’s study (2002) that this situation is known to be new to students from Orhan 364 traditional learning methods to which they are accustomed (Perry, 2002). They prefer direct contact and the direct transfer of knowledge in face-to-face lectures. In this context, because students cannot get used to learning with a constructive approach spontaneously and suddenly, it may be helpful to conclude that learning environments designed with a constructive approach should be presented more frequently to students. LMSs assist lecturers in delivering textbooks, texts, or multimedia tools to their students. This function is seen as a one-way process while facilitating information between the tutor and the learner and replacing previously used distribution tools, such as copiers, with less functioning tools. However, LMSs allow lecturers to be supported in various ways. For example, it can link materials to online use of course support and a calendar so that certain documents can only be viewed at a particular time, for example, if an exam is passed. The most used feature is that it allows online interaction. In this case, many learning platforms allow many modern LMSs users to interact and collaborate. As found in these studies (Garrote & Pettersson, 2007; Garrote & Pettersson, 2011), the results confirmed that lecturers mostly use LMSs to provide access to course materials to their students. Only very few students use the online interaction function regularly. This determination is important because the online interaction function of LMSs does not affect the pedagogical or didactic formation of the lessons. Online interaction provides many possibilities that positively affect and improve the learning process. However, the perceived value of online exchange may depend on what view of "knowledge" we adopt. In this context, it is understood that knowledge can be realized in a cognitive process by interaction with other people when we look at the constructivist view of knowledge and the learning approach in education. In this case, if the lecturers and learners stick to this view, they can take full advantage of collaborative work and online interaction. This perspective includes general competencies such as problem-solving and social skills. And it goes further than one from the point of view that provides the delivery and evaluation of a course’s materials. In this respect, these tools support interaction and collaborative work. And then, it is logical to expect that they can improve the learning process. Many academics would agree that this development should be a desired outcome for the education world, industry, and society. However, there are also demands from the industry and government that higher education should provide new jobs and be justified by economic growth (Beach, 2013; Lorenz, 2006). Nevertheless, Lorenz (2006) argues that the Bologna process will push higher education to be perceived as a marketable commercial practice. This situation is a chance for universities searching for academic freedom and money resources, by the importance of seeking knowledge and promoting democratic values and constructive freedom of expression (Karran, 2009). The process will inevitably force us to focus more on testable knowledge specific to the subject areas in response to government and business demands. As Kromydas (2017) found in his study, this situation has turned people away from their primary purpose in education, making them a tool in commercial development. As can be seen in Table 7, for discussing the course topics with different programs, it can be seen that the majority (73.8%) of students use another program because they are more knowledgeable of digital and technology environments. The results obtained in this study are compatible with the literature (Munoz-Organero, Munoz-Merino & Kloos, 2010; Wang, Woo, Quek, Yang & Liu, 2011). When examining the data in Table 8, half (50.0%) of the lecturers answered “never,” and the minority (21.4%) of lecturers gave positive feedback for encouraging their students to participate in online discussions. It is seen that lecturers do not nurture and engage their International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 365 students in online discussions (35.7%) in Table 9. When the studies in this field are analyzed, it is seen that online discussions take place in education, and LMSs are starting to be used by lecturers and learners. However, it can also be observed in some studies that student participation where online discussions are a condition of the course is higher than in online discussions are optional. In a similar study, Williams and Pury (2002) reported that the students in the group of compulsory students participated in more discussions, read more messages, and wrote more than those in the group of not obligatory. Findings in line with the usage of LMSs confirmed that lecturers accept that such learning platforms are now part of universities. In this context, before starting online discussions, it is necessary to consider what kind of performance is expected in terms of content, style, frequency of participation, and which language will be used. For this reason, the lecturers should determine their criteria before using the LMS since it would be helpful to give an evaluation criterion that indicates the grade value of each of these (Rovai, 2007). 7. Conclusion In recent years, information technologies have been developing rapidly to increase the success of education systems worldwide. Educational activities have now moved out of the institutions to virtual platforms where students can easily access information when needed. Teaching has now been seen as a measurement and evaluation technique. It has become a structure in which electronic evaluation is made immediately after the information is given, far beyond the classical examination and assessment methods. This rapid change in the informatics world shows that the process is no longer irreversible. It opens the way for fast and easy access to information, which is now indispensable in today’s world. Thus, the doors of creating the "intellectual person" have been opened wide. And this is also possible with LMSs. Although there are a lot of LMSs in number at our age, there is no single one that "can meet all needs." Therefore, selected LMSs should be examined with their general characteristics, compared with each other, and the one that best meets the needs of the institution that will use the application should be selected. The use of LMS enables institutional and individual development. Choosing an LMS should be done with great care since it brings a radical change and is very costly. In this case, LMSs should be analyzed with their general features and should be selected considering the best matches the structure of the institution should be selected. For this, appropriate strategies should be determined, and the needs should be analyzed in the most detailed way. Particular attention should be paid to the fact that it is easy to use, flexible, suitable for development, accessible, compatible with other system contents, reliable, and capable of serving 24 hours (Hall, 2002). There are many reasons to use LMSs in the education process in many universities. It plays an important role, mainly due to the increased demand for access to higher education and increased competition for students coming to universities with high media literacy and related expectations. However, to understand the importance and logic of LMSs, it is necessary to see how technology will be used in the future, how it will open up in education, and how the needs of future learners and today’s learners will differ. The “e” in front of e-Learning will likely be removed in the future and learning and teaching procedure will be integrated with the electronic environment. In this context, LMSs will be important in every way in education at universities. At the same time, it should be borne in mind that using LMS in higher education requires a basic understanding of computer science in order to enable effective digitally supported knowledge acquisition, management, and distribution. Because for many of the academics who took their university education in the 1980s, it was a matter, of course, to explain to the students the worldwide body of knowledge of organized concepts in library science. Orhan 366 In conclusion, it can be said that the lecturers in this study were reluctant to use LMS in their lessons. However, the use of LMS systems has become crucial due to the interruption of face-to-face education, especially during the Covid-19 pandemic. It is necessary to identify the deficiencies first and then develop the LMS system since the Covid-19 pandemic period is continuing rapidly. But, in the near future, it is predicted that web-based education can be used as an alternative method planned within the scope of LMS in the near future compared to face- to-face training. In this respect, current research results are thought at a level for setting light, guiding, and contributing to the experiences and evaluations of lecturers from different departments. International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 367 References Akdur, T. E. (2017). Turkey Country Report on ICT in Education. European Schoolnet. Retrieved from http://www.eun.org Alharbi, S. & Drew, S (2014). Using the technology acceptance model in understanding academics’ behavioural intention to use learning management systems. International Journal of Advanced Computer Science and Applications, 5(1), 143-155. Alkan, C. (1987). Öğrenme – öğretme süreçleri ilkeleri. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi (JEFS), 20(1), 209-229. Alyaz, Y., Ş. B. Göktalay, Ş. B & Gürsoy, E. (2014). Öğretim Yönetim Sistemi Ukey’in Kullanılabilirlik Değerlendirmesi ve Öğretmen Adaylarının Görüşleri. Eğitim ve Öğretim Araştırmaları Dergisi 3(37), 392-408. Anastasiadou, S. D. (2010). The development and validation of a new scale for monitoring students’ attitudes to learning statistics with technology. 7th Congress of the European Society in Mathematics Education. Reszow, Poland. Anastasiadou, S. D. (2011). Reliability and validity testing of a new scale for measuring attitudes toward learning statistics with technology. Acta Didactica Napocensia 4(1), 1- 10. Arroway, P., Davenport, E., Guangning X. & Updegrove, D. (2010). EDUCAUSE core data service fiscal year 2009 summary report. EDUCAUSE, White Paper. Ateş, H. (2020). Using Information Systems and Technologies in Higher Education Institutions. Int. J. Res.-GRANTHAALAYAH, 7(10), 222-232.Aydın, C. H. (2003). Uzaktan eğitimin geleceğine ilişkin eğilimler. Elektrik Mühendisliği Odası Dergisi, 419, 28-36. Retrieved from http://www.emo.org.tr/ekler/7e8f8e5982b3298_ek.pdf?dergi=327 Bailey, G. D. (1992). Wanted: A road map for understanding integrated learning systems. Educational Technology 32(9), 3-5. Beach, D. (2013). Changing higher education: converging policy-packages and experience of changing academic work in Sweden. Journal of Education Policy, 28(4), 517-533. doi:10.1080/02680939.2013.782426 Becker, H. J. (1992). A model for improving the performance of integrated learning systems: Mixed individualized/group/whole class lessons, cooperative learning, and organizing time for teacher-led remediation of small groups. Educational Technology 32(9), 6-15. Bland, J. M. & Altman, D. G. (1997). Statistics notes: Cronbach's alpha. Bmj 314(7080), 572. Blin, F. & Munro, M. (2008). Why hasn’t technology disrupted academics’ teaching practices? Understanding resistance to change through the lens of activity theory. Computers & Education, 2008. 50(2), 475-490. doi:10.1016/j.compedu.2007.09.017 Bougie, R. & Sekaran, U. (2019). Research methods for business: A skill building approach. John Wiley & Sons. Borchert, T., Fritzenberg, M. & Schlöffel, R. (2017). Blended Learning im Sportstudium. Zwischen universitärer Persistenz und berufsfeldorientierter Kompetenzentwicklung. P. Neumann & E. Balz. Sportlehrerausbildung heute–Ideen und Innovationen, 49-59. http://www.eun.org/ https://doi.org/10.1080/02680939.2013.782426 https://doi.org/10.1016/j.compedu.2007.09.017 Orhan 368 Browne, T., Jenkins, M. & Walker R. (2006). A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. Interactive Learning Environments 14(2), 177-192. Brush, T. A., Armstrong, J., Barbrow, D. & Ulintz, L. (1999). Design and delivery of integrated learning systems: Their impact on student achievement and attitudes. Journal of Educational Computing Research 21(4), 475-486. Bursa Uludag University. Retrieved from www.uludag.edu.tr Bush, M. D. & Mott, J. D. (2009). The Transformation of Learning with Technology: Learner- Centricity, Content and Tool Malleability, and Network Effects. Educational Technology, 49(2), 3-20. Cavus, N. & Ala’a, M. M. (2009). Computer aided evaluation of learning management systems. Procedia-Social and Behavioral Sciences 1(1), 426-430. Cheng, C. K., Paré, D. E., Collimore, L.-M. & Joordens, S. (2011). Assessing the effectiveness of a voluntary online discussion forum on improving students’ course performance. Computers & Education, 56(1), 253-261. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. psychometrika 16(3), 297-334. Cronbach, L. J. (1984). A research worker's treasure chest. Multivariate behavioral research 19(2-3), 223-240. Cronjé, J. (2006). Paradigms regained: Toward integrating objectivism and constructivism in instructional design and the learning sciences. Educational Technology Research and Development, 54(4), 387-416. doi:10.1007/s11423-006-9605-1 Çağlayan, U. & Bener, A. (2006). Avrupa birliği sürecinde Türkiye’de bilişim ve telekomünikasyon teknolojileri sektörü üzerine görüş ve öneriler. TÜSİAD, 6. Dictionary. (2022). Learning Management System. Retrieved from https://www.dictionary.com/browse/learning-management-system Duran, N., Önal, A. & Kurtuluş, C. (2006). E-öğrenme ve kurumsal eğitimde yeni yaklaşım öğrenim yönetim sistemleri, Bilgi Teknolojileri Kongresi IV; Akademik Bilişim, Bildiriler Kitabı, 97-101. Retrieved from URL: http://ab.org.tr/ab06/bildiri/165.pdf Dutton, W. H., Hope Cheong, P. & Park, N. (2004). The social shaping of a virtual learning environment: The case of a university-wide course management system. The Electronic Journal of e-Learning, 2(1), 69-80. Ehlers, U.-D. (2011). Qualität im e-learning aus Lernersicht. 327 Seiten. ISBN: 978-3-531- 93070-1. Wiesbaden: Springer. Eisinga, R., Grotenhuis, M.T. & Pelzer, B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? International journal of public health 58(4), 637-642. Elsharif, T. A. (2019). The Application of Quality Costs Accounting in Industrial Companies in Benghazi Libya. Open Access Library Journal 6(05), 1. Federico, P.-A. (2000). Learning styles and student attitudes towards various aspects of network-based instruction. Computers in Human Behavior, 16(4), 359-379. doi:10.1016/S0747-5632(00)00021-2 https://doi.org/10.1007/s11423-006-9605-1 https://www.dictionary.com/browse/learning-management-system https://doi.org/10.1016/S0747-5632(00)00021-2 International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 369 Fındık, D. & Özkan, S. (2010). Identifying success factors for WBLMS use by instructors of engineering departments. IEEE Frontiers in Education Conference (FIE). doi:10.1109/FIE.2010.5673582 Fox, B. (2007). Teaching Through Technology: Changing Practices in Two Universities. International Journal on E-Learning, 6(2):187-203. Waynesville, NC USA: Association for the Advancement of Computing in Education (AACE). Retrieved from https://www.learntechlib.org/primary/p/19810/ Garrote Jurado, R. & Pettersson, T. (2007). Lecturers’ attitudes about the use of learning management systems in engineering education: A Swedish case study. Australasian Journal of Educational Technology, 23(3), 327-349. doi:10.14742/ajet.1256 Garrote Jurado, R. & Pettersson, T. (2011). The use of learning management systems: A Longitudinal Case Study. eleed, 8(1),1-11. urn:nbn:de:0009-5-31456. Retrieved from https://eleed.campussource.de/archive/8/3145 Garrote Jurado, R., Pettersson, T. & Arazola Ruano, C. (2019). European Lecturers’ perceptions of Interactive Tools in Learning Management Systems. 12th International Conference of Education Research and Innovation, Seville, Spain. Gilhooly, K. (2001). Making e-learning effective. Computerworld 35(29), 52-53. Gliem, J. A. & Gliem, R. R. (2003). Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales, Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education. Goetz, M. (2020). Distance Learning in der COVID-19 Krise: Ein Praxischeck. Medienimpulse 58(02), 1-21. Hair, J., Black, W., Babin, B., Anderson, R. & Tatham, R. (2014). Multivariate data analysis. Pearson Education Limited. England. Hall, J. L. (2002). Assessing learning management systems. Chief Learning Officer, Retrieved from https://www.chieflearningofficer.com/2002/12/11/assessing-learning- management-systems/ Hao, Y. (2016). Middle school students' flipped learning readiness in foreign language classrooms: Exploring its relationship with personal characteristics and individual circumstances. Computers in Human Behavior 59, 295-303. Hopperton, L. G. (1998). Computer Conferencing and College Education, College Quarterly, 5(2). Herse, P., & Lee, A. (2005). Optometry and WebCT: A student survey of the value of web- based learning environments in optometric education. Clinical and Experimental Optometry, 88(1), 46–52. doi:10.1111/j.1444-0938.2005.tb06663.x Irwin, C. & Berge, Z. (2006). Socialization in the Online Classroom. e-Journal of Instructional Science and Technology, 9(1), 1-7. Jemain, A. A., Al-Omari, A. & Ibrahim, K. (2007). Multistage median ranked set sampling for estimating the population median. Journal of Mathematics and Statistics 3(2), 58-64. Kanuka, H. & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. The Journal of Distance Education / Revue de l’ducation Distance, 13(1), 57-74. Athabasca University Press. Retrieved from https://www.learntechlib.org/p/86791/. https://www.researchgate.net/deref/http%3A%2F%2Fdx.doi.org%2F10.1109%2FFIE.2010.5673582?_sg%5B0%5D=BLy7niRhsKhNo0y-nxYmJNrfsY76LIWyW_zFW0iQcPCatDKPR9NBa0cPmiFiw74bnlXOhjvn9og7xQy8JcopNCjJsQ.mtW5sjqzliU42PRGyT0XYMKr5kYLqkTS7SV9cPY_dRDxDGswq9JEmnlDqfUDaB6BXrLhVLT9rXvgpawprSnLaA https://doi.org/10.14742/ajet.1256 https://eleed.campussource.de/archive/8/3145 https://www.chieflearningofficer.com/2002/12/11/assessing-learning-management-systems/ https://www.chieflearningofficer.com/2002/12/11/assessing-learning-management-systems/ https://doi.org/10.1111/j.1444-0938.2005.tb06663.x https://www.learntechlib.org/p/86791/ Orhan 370 Karran, T. (2009). Academic freedom: in justification of a universal ideal. Studies in Higher Education, 34(3), 263-283. doi:10.1080/03075070802597036 Kerres, M. (2018). Mediendidaktik: Konzeption und Entwicklung digitaler Lernangebote. Oldenbourg, De Gruyter. Kerres, M. (2020). Against all odds: Education in Germany coping with Covid-19. Postdigital Science and Education 2(3), 690-694. Kreidl, C. (2011). Akzeptanz und Nutzung von E-Learning-Elementen an Hochschulen. Gründe für die Einführung und Kriterien der Anwendung von E-Learning. Münster: Waxmann. Kromydas, T. (2017). Rethinking higher education and its relationship with social inequalities: past knowledge, present state and future potential. Palgrave Communications, 3(1), 1- 12. doi:10.1057/s41599-017-0001-8 Kultusministerkonferenz (2016). Bildung in der digitalen Welt. Bildung in der digitalen Welt: Strategie der Kultusministerkonferenz Berlin. Kurnaz, E. & Serçemeli, M. (2020). COVID-19 Pandemi Döneminde Öğrencilerin Uzaktan Eğitim ve Uzaktan Muhasebe Eğitimine Yönelik Bakış Açıları Üzerine Bir Araştırma. Uluslararası Sosyal Bilimler Akademi Dergisi(3), 262-288. Ladyshewsky, R. K. & Gardner, P. (2008). Peer assisted learning and blogging: A strategy to promote reflective practice during clinical fieldwork. Australasian Journal of Educational Technology, 24(3), 241-257. Leon, A. C.,. Marzuk, P. M & Laura, P. (1995). More reliable outcome measures can reduce sample size requirements. Archives of general psychiatry 52(10), 867-871. Lewis, T., Marginson, S. & Snyder, I. (2005). The network university? Technology, culture and organisational complexity in contemporary higher education. Higher Education Quarterly 59(1), 56-75. Likert, R. (1931). A technique for the measurement of attitudes. Archives of Psychology. New York: Columbia University Press. Lonn, S. & Teasley, S. D. (2009). Saving time or innovating practice: Investigating perceptions and uses of learning management systems. Computers & Education, 53, 686-694. doi:10.1016/j.compedu.2009.04.008 Lorenz, C. (2006). Will the universities survive the European integration? Higher Education Policies in the EU and in the Netherlands before and after the Bologna Declaration. Sociologia internationalis, 44(4), 1-28. Malikowski, S. R., Thompson, M. E. & Theis, J. G. (2007). A model for research into course management systems: Bridging technology and learning theory. Journal of Educational Computing Research, 36(2),149-173. doi:10.2190/1002-1T50-27G2-H3V7 Metzner, J., Bartosch, U., Vogel, M., Schroll, A.-L., Rademacher, M. & Neuhausen, H. (2019). Was bedeutet Hochschule im digitalen Zeitalter? Eine Betrachtung des Bildungsbegriffs vor den Herausforderungen der Digitalisierung. Arbeitspapier, 50. Berlin: Hochschulforum Digitalisierung. Meyer, P. (2016). Lernmanagement-Systeme an Hochschulen: Resümee des Themenspecials. Retrievied from https://www.e-teaching.org/news/eteaching_blog/themenspecial- lernmanagement-systeme-ein-resuemee https://doi.org/10.1080/03075070802597036 https://doi.org/10.1057/s41599-017-0001-8 https://doi.org/10.1016/j.compedu.2009.04.008 https://doi.org/10.2190/1002-1T50-27G2-H3V7 https://www.e-teaching.org/news/eteaching_blog/themenspecial-lernmanagement-systeme-ein-resuemee https://www.e-teaching.org/news/eteaching_blog/themenspecial-lernmanagement-systeme-ein-resuemee International Online Journal of Education and Teaching (IOJET) 2022, 10(1), 348-372. 371 Mitchell, V. W. (1998). Improving mail survey responses from UK academics: Some empirical findings. Assessment and Evaluation in Higher Education 23(1), 59-70. Mogus, A.M., Djurjevic, I. & Suvak, N. (2012), The impact of student activity in a virtual learning environment on their final mark. Active Learning in Higher Education, 13(3), 177-189. doi:10.1177/1469787412452985 Munoz-Organero, M., Munoz-Merino, P. J. & Kloos, C. D. (2010). Student behavior and interaction patterns with an LMS as motivation predictors in e-learning settings. IEEE Transactions on Education, 53(3), 463-470. doi:10.1109/TE.2009.2027433 Murphy, E. (2004). Recognising and promoting collaboration in an online asynchronous discussion. British Journal of Educational Technology, 35 (4), 421-431. doi:10.1111/j.0007-1013.2004.00401.x Murphy, E. & Loveless, J. (2005). Students' self analysis of contributions to online asynchronous discussions. Australasian Journal of Educational Technology 21(2), 155- 172. Müller, P. (2018). Was Lernmanagementsysteme heute leisten können müssen. Retrieved from www.elearning-journal.de/index.php Naidu, S. (2006). E-learning: A guidebook of principles, procedures and practices. Commonwealth Educational Media Centre for Asia (CEMCA). Newberg, L. A., Rouse III, R. O. & Kruper, J. A. (1994). Integrating the world-wide-web and multi-user domains to support advanced network-based learning environments. ED- MEDIA 95 – Conference Proceedings, 494-499. Retrieved from http://www.rpi.edu/~newbel/publications/NewbergRouseKruper1994.pdf Nunnally, J. C. (1978). Psychometric theory. New York, McGraw-Hill Book Company. Parr, J. & Fung, I. (2000). A review of the literature on computer-assisted learning, particularly integrated learning systems, and outcomes with respect to literacy and numeracy. Ministry of Education: Wellington. Paulsen, F. M. (2002). Online Education Systems: Discussion and definition of terms. NKI Distance Education, 1-8. Perry, G.Y. (2002). The use of online video case studies and teacher education. Doctoral thesis. Pepperdine University. Retrieved from https://www.learntechlib.org/p/121082/ Persike, M. & Friedrich, J.-D. (2016). Lernen mit digitalen Medien aus Studierendenperspektive. Berlin: Hochschulforum Digitalisierung. Rovai, A.P. (2007). Facilitating online discussions effectively. The Internet and Higher Education,10(1), 77–88. doi:10.1016/j.iheduc.2006.10.001 Rubin, B., Fernandes, R., Avgerinou, M. D. & Moore, J. (2010). The effect of learning management systems on student and faculty outcomes. Internet and Higher Education, 13:82-83. doi:10.1016/j.iheduc.2009.10.008 Santos, J. R. A. (1999). Cronbach’s alpha: A tool for assessing the reliability of scales. Journal of extension 37(2), 1-5. Schnell, R., Hill, P. B. & Esser, E. (1999). Methoden der empirischen Sozialforschung. München: Oldenbourg Verlag. Sekaran, U. & Bougie, R. (2016). Research methods for business: A skill building approach, john wiley & sons. https://doi.org/10.1177/1469787412452985 https://doi.org/10.1109/TE.2009.2027433 https://doi.org/10.1111/j.0007-1013.2004.00401.x https://www.learntechlib.org/p/121082/ https://doi.org/10.1016/j.iheduc.2006.10.001 https://doi.org/10.1016/j.iheduc.2009.10.008 Orhan 372 Smith, G. (2008). Does gender influence online survey participation?: A record-linkage analysis of university faculty online survey response behavior. ERIC Document Reproduction Service No. ED 501717, SJSU ScholarWorks, 1-22 Szabo, M. (2002). Cmi theory and practice: Historical roots of learning managment systems. E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Association for the Advancement of Computing in Education (AACE). Tecim, V. & Gökşen, Y. (2009). Bilişim teknolojilerinin üniversitelerde etkin kullanimi üzerine bir çalişma. Journal of Yasar University 4(14), 2237-2256. Tosun, E. K. (2008). Türkiye’de Bilgi ve İletişim Teknolojilerinin (BIT) Durumu ve E- Dönüşüm Projesi. Paradoks Ekonomi Sosyoloji ve Politika Dergisi 4(2), 1-5. Ubell, R. (2000). Engineers turn to e-learning, IEEE Spectrum, 37(10), 59-63. doi:10.1109/6.873919 UKEY. Retrieved from http://www.uludag.edu.tr/dosyalar/bilgiislem/ukey/UKEY_KK.pdf Wikipedia (2022). Learning Management System (LMS). Retrieved from https://tr.wikipedia.org/wiki/%C3%96%C4%9Frenme_y%C3%B6netim_sistemleri_(L MS) Watson, W. R. & Watson, S. L. (2007). An Argument for Clarity: What are Learning Management Systems, What are They Not, and What Should They Become?. TechTrends, 51(2), 28-34. doi:10.1007/s11528-007-0023-y Wang, Q., Woo, L. H., Quek, C. L., Yang, Y. & Liu, M. (2011). Using the Facebook group as learning management system: An exploratory study. British Journal of Educational Technology, 43(3), 428-438. doi:10.1111/j.1467-8535.2011.01195.x Wendeborn, T., Schneider, A. & Karapanos, M. (2018). Lernplattformen oder Content- Halden? Learning-Management-Systeme in der Schulpraxis. Workshop Gemeinschaften in Neuen Medien (GeNeMe) 2018, TUDpress. West, Richard. E., Waddoups, G. & Graham, C. R. (2007). Understanding the experiences of instructors as they adopt a course management system. Educational Technology Research and Development, 55(1), 1-26. doi:10.1007/s11423-006-9018-1 Williams, S. & Pury, C. (2002). Student attitudes toward and participation in electronic Discussions, International Journal of Educational Technology, 3(1). Wilson, G. & Stacey, E. (2004). Online interaction impacts on learning: Teaching the teachers to teach online. Australasian journal of educational technology 20(1), 33-48. Wright, K. B. (2005). Researching Internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of computer-mediated communication 10(3), JCMC1034. Yarusso, L. (1992). Constructivism vs. objectivism. Performance + Instruction, 31(4), 7-9. doi:10.1002/pfi.4170310404 https://doi.org/10.1109/6.873919 https://tr.wikipedia.org/wiki/%C3%96%C4%9Frenme_y%C3%B6netim_sistemleri_(LMS https://tr.wikipedia.org/wiki/%C3%96%C4%9Frenme_y%C3%B6netim_sistemleri_(LMS https://doi.org/10.1007/s11528-007-0023-y https://doi.org/10.1007/s11423-006-9018-1 https://doi.org/10.1002/pfi.4170310404