Özüdoğru, M., & Aksu, M. (2019). A new approach in higher education: the perceptions of pre-service teachers related to flipped learning. International Online Journal of Education and Teaching (IOJET), 6(2). 295-316 http://iojet.org/index.php/IOJET/article/view/586 Received: 18.12.2018 Received in revised form: 25.01.2019 Accepted: 18.02.2019 A NEW APPROACH IN HIGHER EDUCATION: THE PERCEPTIONS OF PRE- SERVICE TEACHERS RELATED TO FLIPPED LEARNING Research Article Melike Özüdoğru Manisa Celal Bayar University ozudogru123@gmail.com Meral Aksu Middle East Technical University aksume@metu.edu.tr Melike Özüdoğru is Dr. Reseach Assistant in Educational Sciences, Curriculum and Instruction Department in Manisa Celal Bayar University. She is interested in integration of technology to teaching and learning process, teaching mathematics, e-learning, instructional design, and curriculum evaluation. Meral Aksu was Prof. Dr. in Educational Sciences, Curriculum and Instruction Department in Middle East Technical University. She has just retired. Copyright by Informascope. Material published and so copyrighted may not be published elsewhere without the written permission of IOJET. http://iojet.org/index.php/IOJET/article/view/586 mailto:ozudogru123@gmail.com mailto:aksume@metu.edu.tr http://orcid.org/0000-0002-7478-3576 http://orcid.org/0000-0003-4963-3901 International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 295 A NEW APPROACH IN HIGHER EDUCATION: THE PERCEPTIONS OF PRE-SERVICE TEACHERS RELATED TO FLIPPED LEARNING Melike Özüdoğru ozudogru123@gmail.com Meral Aksu aksume@metu.edu.tr Abstract The purpose of this study was to develop and implement flipped learning materials in the Principles and Methods of Instruction course and investigate the perceptions of pre-service teachers about flipped learning. This study was conducted in the fall semester of 2017-2018 for 11 weeks at a public university located in the Aegean Region in Turkey. This study employed a mixed method research design. There were 30 pre-service teachers in the flipped learning group. In the current study, student questionnaire and interview schedule were implemented. The data were analyzed by using descriptive statistical analysis techniques. For the analysis of the data, SPSS 22.0 was used and alpha level was determined as .05. The data obtained from the interviews were analyzed by using both the content and descriptive analysis techniques. The findings of the study showed that pre-service teachers mostly had positive perceptions about flipped learning processes and materials. They agreed that flipped learning supported and facilitated learning was effective in order to practice theoretical knowledge and contributed to personal development and self-enhancement. Their perceptions were examined and discussed in detail. Keywords: Flipped learning, pre-service teacher education, videos 1. Introduction In traditional pre-service teacher training classrooms students come to class to learn the content of lectures from instructors. They take notes, work on assignments, projects, tasks and other activities, usually on their own, outside of the class (Bergman & Sams, 2012; Hamdan, McKnight, McKnight, & Arfstrom, 2013; Talbert, 2012). In this type of teaching, the role of instructors is mainly the transfer of information (Love, Hodge, Grandgenett & Swift, 2014; Talbert, 2014). Foertsch et al. (2002) stated that the traditional instruction was an ineffective way of teaching because of including a passive one-way flow of information from professor to students which is according to Bloom‟s taxonomy at the lowest cognitive level. Moreover, in traditional classes, students have to perform higher level tasks outside of classes on their own and remote from their instructors‟ help. Hence, it seems that changing the order of instruction might be an improvement for pre-service teacher education classes. For this purpose, an active teaching approach, flipped learning was proposed (Davies et al., 2013; Missildine et al., 2013; Sams & Bergmann, 2013; Talbert, 2012). Lage, Platt, & Treglia (2000) defined flipped learning “events that have traditionally taken place inside the classroom now take place outside the classroom and vice versa” (p. 32). It is also known as flipped classroom, inverted classroom, flipped lessons, flipped learning or flipped thinking (Sams & Bergman, 2013). The idea of flipped learning is that instead of allocating time to present a concept through lecturing, the instructor can use limited and Özüdoğru&Aksu 296 significant class time for more involving activities and group work. The fundamental principle of the flipped learning is that students begin to interact with new concepts outside of the traditional classroom. They study through videos, screencasts or podcasts when and where they feel most comfortable by considering their own learning speed and then they apply what they have learned in the classroom (Abeysekera & Dawson, 2015; Bergman & Sams, 2012; Bormann, 2014; Mason, Shuman, & Cook, 2013; Milman, 2012; Phillips & Trainor, 2014; Talbert, 2014). In this way, face to face part of the class can be allocated for discussions, answering students‟ questions which were uncovered during pre-class preparation and engaging collaborative learning activities (Ziegelmeier & Topaz, 2015) In the face-to-face part of the course, students can present a summary of their reflections or face-to-face class time can begin with a short quiz about the content of online lecture. Also, students‟ pre-class preparation can be assessed thorough in-class student response systems like Kahoot or Socrative, peer discussions and presentations, which may require the utilization of smartphone apps and tablets to answer clicker questions in order to provide immediate feedback about misconceptions or learning gaps as formative assessment (Lage et al., 2000; O‟Flaherty & Phillips, 2015; Talbert, 2012). During flipped learning, the face-to- face class time is used for interactive exercises in which the students can apply their new knowledge under the guidance of their instructor together with their peers (Foertsch et al., 2002; (Munir, Baroutiana, Younga, & Carter, 2018). In other words, in flipped learning, in- class activities involve focusing on and internalizing the material with the direct help of peers and the instructor. The consequence can be a more interactive, effective, productive and more student-centered classroom instead of a passive traditional classroom (Talbert, 2012). It can be seen that flipped learning requires a shift from a teacher-centered instruction to a student- centered one, a change in pedagogical practices and the use of technology that focuses on individualized and active learning structures such as differentiated learning, mastery learning, cooperative learning and collaborative learning in which students are engaged in their learning. In terms of videos, they were stated to be divided into brief parts, the content should be shorter and plain than any other course book (Bergmann & Sams, 2012; Caudill, 2014; Enfield, 2013; Turan, 2015). In the study of Caudill (2014), pre-service teachers in a flipped Applied Child Development course stated teachers‟ keeping the lecture videos down to a minimum of ten minutes and teaching the most important things were better than long lecturing. Ceylaner (2016) found that some ninth grade students checked the duration of videos before watching them and watched the videos by skipping some parts which were more than five minutes. In the study conducted by Zappe et al. (2009), undergraduate architectural engineering students expressed their ideas that according to them the optimum video length to be around 20 minutes. In the current study, around 21 minutes videos were divided into two or three parts to be managed by pre-service teachers easily. In the literature, most of the studies investigated the effect of flipped learning on students‟ achievement together with other cognitive and affective variables. It was determined that flipped learning improved student learning and achievement (Wilson, 2013; Talley & Scherer, 2013; Tune et al., 2013; Davies et al, 2013; Murphree, 2014; Talbert, 2014). In addition, Mason et al. (2013) stated that flipped learning allowed the instructor to include more content and provide better performance of students on quizzes and exams than traditional approach in a Control Systems course in the Department of Mechanical Engineering. On the other hand, some of the studies indicated insignificant differences in student learning (Clark, 2013; Findlay-Thompson & Mombourquette, 2014; McLaughlin et al., 2013; Morin et al., 2013). International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 297 In addition to these, learning with videos provides students with access to the course content independent of the space (Alsancak-Sırakaya, 2015; Bergman & Sams, 2012; Guc, 2017). In this way, they had the opportunity of making up the courses by themselves. Similarly, in the study conducted by Alsancak-Sırakaya (2015) senior pre-service teachers taking the Scientific Research Methods course and studying in the Guidance and Psychological Counseling department stated that when they were absent in the course, they had the chance of watching the videos over and over, which might have increased the level of learning similar to current study. Moreover, in the study conducted by Guc (2017), secondary school students stated that they could not learn the subjects taught in the first lesson of the day because of coming to school quite early and being sleepless in other courses taught according to traditional principles. However, they stated that during flipped learning they could study productively and learn when they felt physically and emotionally ready. Flipped learning provides differentiation by customizing the curriculum for different learning styles to achieve mastery of learning objectives as also stresses by Fulton (2012). Similarly, Mason et al. (2013) stated that flipped learning is proper to teach course material with different teaching methods and involve students with various learning styles like visual, audial, verbal, active or reflective. In this way, learning is personalized for each pre-service teachers since they have the control of their own learning. While quick learners are able to fast forward and work on additional resources to go deeper into the topics (Caudill, 2014), slow learners can rewind to hear the online lectures again, listen as much as they need, pause to reflect on what is being said and view the lectures on a mobile device rather than in a fixed location. O‟Flaherty & Phillips (2015) stated one of the multiple justifications for flipped learning that it was seen as a way to promote more participatory and empowering learning experiences for students. It includes multiple communication opportunities between the instructor and students, hence students can ask questions and instructors give immediate feedback which facilitate learning (Ziegelmeier & Topaz, 2015). For example, the inclusion of social media and the online discussion boards in which students can ask questions to the instructor and their peers at any time and get help from both sides diminish the feeling that students are being made to learn on their own (Talbert, 2014). The results of many studies indicated positive perceptions related to flipped learning (Bergman & Sams, 2012; Berrett, 2012; Bishop & Verleger, 2013; Cibik, 2017; Lee, Lim, & Kim, 2016; Kurt, 2017; O‟Flaherty & Phillips, 2015; Reinhardt, 2014; Touchton, 2015; Turan, 2015). On the other hand, the results of the study conducted by Kecskemety and Morin (2014) suggested dissatisfaction with the flipped learning approach for the engineering students and Missildine et al. (2013) reported that nursing students were significantly less satisfied with the flipped learning than traditional instruction. Finally, in the study conducted by Findlay-Thompson & Mombourquette (2014), students were undecided about their responses that they stated both positive and negative ideas about the flipped learning in a business course. Hence, it was thought important to investigate the perceptions of pre- service teachers related to flipped learning. Moreover, the design of courses using the flipped learning involves some challenges. It requires a heavy responsibility of instructors in terms of preparing materials and pre-class assignments (Enfield, 2013). Furthermore, as also stated by Milman (2012), instructors may not produce high quality videos technically or instructionally. Besides, passing forward or backward in the video with long delays in loading, freezing of the video and downloading the videos sometimes take a long time. Hence, it was thought important to investigate the perceptions of pre-service teachers‟ related to videos both technically and in terms of contribution to learning. Özüdoğru&Aksu 298 1.1. The aim of the study and research question The purpose of this study was to develop and implement flipped learning materials in the Principles and Methods of Instruction course and investigate the perceptions of pre-service teachers related to classroom environment. Based upon the main purpose of the study, the following research question was proposed: What are the perceptions of pre-service teachers about flipped learning? 2. Method In this part, research design participants of the study, data collection instrument, data collection procedures and data analysis were explained. 2.1. Research design In this study, both quantitative and qualitative data were collected. Hence, this study employed mixed method (Creswell, 2012). In this study, quantitative data were collected through survey research design (Cohen, Manion & Morrison, 2007). In this type, data were collected at a particular point in time in order to describe the characteristics, opinions and conditions that exist in the population (Fraenkel & Wallen, 2009). Then, the perceptions of pre-service teachers about the videos used in flipped learning were asked. The responses of pre-service teachers were categorized under themes. 2.2. Participants of the study The surveys were implemented to 30 pre-service teachers. Among the 30 pre-service teachers, 25 (83.3%) of them were female and 5 (16.7%) of them were male. In the qualitative part of the study, twelve pre-service teachers were selected according to maximum variation sampling method in order to ask their perceptions about videos used in flipped learning. The sample was selected in order to represent the diversity of perspectives or characteristics (Fraenkel & Wallen, 2009; Gall, Gall, & Borg, 2003). In this way, the strength and richness of the data, their applicability and interpretation were ensured better (Cohen et al., 2007). The interviewees were selected purposefully based on their achievements in the Principles and Methods of Instruction course and gender. Among the twelve pre-service teachers nine of them were female and three of them were male. The codes of pre-service teachers and their grades were shown in Table 1. Table 1. The codes of pre-service teachers chosen for interviews and their course grades Code of the Pre-Service Teachers Achievement Test Scores Final Grade Ga (High Achiever) 83 90 Nu (High Achiever) 63 82 Ra (High Achiever) 75 86 En (High Achiever) 78 82 Me (Low Achiever) 65 73 Fa (High Achiever) 78 87 En (Medium Achiever) 70 79 Mu (High Achiever) 85 88 Bu (High Achiever) 78 87 Ha (Low Achiever) 55 75 International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 299 Fa (Medium Achiever) 63 78 Me (Medium Achiever) 70 79 Pre-service teachers who have final grades between 70 and 75 were accepted as having low achievement score, the pre-service teachers who have final grades between 76 and 80 were accepted as having medium achievement score and pre-service teachers who have final grades over 81 were accepted as having high achievement score. The codes were comprised of the first and second characters of their names and assigned achievement level of pre- service teachers. 2.3. Data collection instruments and procedures The study was conducted during the fall semester of 2017-2018 at a public university located in the Aegean Region. The study was implemented at the Classroom Teaching Department-Elementary Education Division in the Principles and Methods of Instruction course which is a three-hour a week course. The course lasted for 15 weeks but except four weeks (one week is orientation week, two weeks are midterm exams, and one week after midterm exams), the treatment process was carried out for 11 weeks. In this study, „Student Questionnaire‟ was implemented to answer the research question. Also, pre-service teachers were asked about the videos used in flipped learning. The perceptions of flipped learning group regarding the video courses, including different habits of watching the videos and their perceptions about the effect of using videos on pre- service teachers‟ learning were asked through student questionnaire. Moreover, the perceptions of pre-service teachers about the flipped learning applications such as whether group studies affected pre-service teachers‟ learning positively, developed self-learning skills, supported and facilitated learning, flipped learning took longer time to learn or not, whether it was perceived as distracting because of being so much activities or not were asked through student questionnaire. The first part of the questionnaire included personal questions and the second part of the questionnaire included 33 items. The categories for the questions 1 to 6 represent (3) always, (2) sometimes, and (1) never, and questions from 7 to 33 represent (5) strongly agree, (4) agree, (3) partially agree, (2) disagree, and (1) strongly disagree. For the pilot study of the questionnaire, 259 pre-service teachers who learn all of the courses through flipped learning at a private university in Turkey were asked to fill the questionnaire. The Cronbach Alpha reliability coefficient of internal consistency of the questionnaire was found .95 and descriptive statistics methods were used to analyze the data. The questionnaire included positively and negatively constructed statements which were adapted from existing instruments or were newly developed (Clark, 2013; Day & Foley, 2006; Enfield, 2013; Gaughan, 2014; Kecskemety & Morin, 2014; Turan, 2015; Zappe, et al., 2009; Ziegelmeier & Topaz, 2015). In order to establish validity, the items were checked to by seven experts (two professor, two associate professors, and four assistant professors) in Curriculum and Instruction Department of Ege University, Hacettepe University, Iowa State University and Middle East Technical University as suggested by Turgut & Baykul (2011). The interviews were recorded after taking the permission of pre-service teachers. They were transcribed and coded. Codes were generated in accordance with the literature about flipped learning (Miles, Huberman, & Saldaña, 2014). Özüdoğru&Aksu 300 During the orientation period, which was the first week of the semester, pre-service teachers were informed about the course applications and materials that would be used during the semester and they were provided with membership from the Facebook, WhatsApp groups and the course management system that is Edmodo. The experimental process was explained briefly below. 2.3.1. The procedures conducted in the experimental and control Groups The experimental group was taught by using flipped learning and pre-service teachers were provided with videos which were developed by the researcher before the course. Pre- service teachers were expected to watch video lessons (approximately 21 minutes), summarize the content of the videos in written form and share them on the Moodle system- Edmodo. Each video lesson included pop-up questions which were provided for experimental group pre-service teachers to test whether they learned the subject or not. In face-to-face part of the course, pre-service teachers were not given additional lecturing but the important points that pre-service teachers did not understand were explained. Then the pre-service teachers in the experimental group were directed to Kahoot or Socrative applications which are online question and answer game activity. The names of the pre- service teachers who took the top three were shared on Facebook group to increase the motivation and engagement of pre-service teachers towards the course. Finally, pre-service teachers completed a group work in the class which required them to reflect on, discuss, and apply what they have learned theoretically in videos. In the flipped learning, by participating in cooperative and collaborative learning activities, pre-service teachers completed different tasks like preparing sample products, lesson plans and micro teaching applications about different techniques. The pre-service teachers uploaded the classroom assignments to Moodle to be evaluated. 2.4. Data analysis In the current study, data were collected and analyzed by using descriptive statistical analysis techniques by indicating means (M) and standard deviations (SD) (Field, 2009; Tabachnick & Fidell, 2007). For the analysis of quantitative data, SPSS 22.0 was used and alpha level was determined as .05. The qualitative data were categorized under themes and sub-themes. 3. Results In this part, the results obtained from student questionnaire and interviews were presented. 3.1. The results obtained from student questionnaire on flipped learning Student questionnaire was administered to the pre-service teachers who were involved in the experimental group in order to learn their habits of watching videos and perceptions about video lessons and flipped learning after the treatment process. The results about the habits of watching videos were shown in Table 2. International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 301 Table 2. Descriptive statistics related to pre-service teachers’ habits of watching videos *The mean scores of items1-6 were evaluated over 3. According to the results of SQ analysis as shown in Table 2, it was found that while watching the videos, pre-service teachers always took notes (86.7%), watched the videos carefully (80%) and watched some parts again (50%). On the other hand, it was found that 43% of the pre-service teachers never wrote questions to be asked in the class. In addition to these, according to perceptions of pre-service teachers, as shown in Table 3, pre-service teachers agreed that learning with videos affected their learning positively by providing the opportunity of turning back and watching some parts again and by making up the courses by themselves (100%). Moreover, it increased their interest in the course because of being enriched with audio visuals (90%), increased interaction with the instructor within the class and out of the class (90%) and provided access to the course content independent of the space (86.6%). On the other hand, pre-service teachers disagreed that learning with videos was inefficient due to technical problems (76.7%) and was more difficult than the methods used in other courses (66.7%). The items that take place in Table 3 are the main properties of flipped learning and pre-service teachers showed their ideas positively about this new approach. Table 3. Frequencies, percentages, mean scores and standard deviations of pre-service teachers’ perceptions of learning with videos Always Sometimes Never M SD f % f % f % 1. I watched the videos carefully. 24 80 6 20 - - 2.80 .41 2. I was not interested in another thing while watching. 12 40 16 53.3 2 6.7 2.33 .61 3. I watched some parts again. 15 50 14 46.7 1 3.3 2.47 .57 4. I took notes while watching. 26 86.7 4 13.3 - - 2.87 .35 5. I wrote questions while watching. 4 13.3 13 43.3 13 43.3 1.70 .70 6. I watched videos piece by piece. 14 46.7 15 50 1 3.3 2.43 .57 Learning with videos Strongly agree Agree Partly agree Disagree Strongly Disagree M SD f % F % f % f % f % 7. was easier. 4 13.3 20 66.7 5 16.7 1 3.3 - - 3.90 .66 8. provided access to the course content independent of the space. 13 43.3 13 43.3 3 10 1 3.3 - - 4.27 .78 9. provided self- learning opportunities. 10 33.3 17 56.7 3 10 - - - - 4.23 .63 10. provided learning when and where I wanted. 16 53.3 8 26.7 6 20 - - - - 4.33 .80 11. facilitated practice in face-to 15 50 13 43.3 2 6.7 - - - - 4.43 .63 Özüdoğru&Aksu 302 *Ite ms 7-18 were eval uate d over 5. According to perceptions of pre-service teachers, as shown in Table 4, in terms of learning contribution, pre-service teachers agreed that flipped learning yielded more learning than the courses taught by other methods (86.7%), supported and facilitated learning and was effective to practice theoretical knowledge (90%). In addition, pre-service teachers agreed that it made learning fun (76.7%) and attracted the attention of pre-service teachers owing to the use of different technologies (63.3%). face courses. 12. increased interaction with the instructor in and out of the course. 14 46.7 13 43.3 3 10 - - - - 4.37 .67 13. affected my learning positively because of having the opportunity to go back and watch some parts again. 20 66.7 10 33.3 - - - - - - 4.67 .48 14. made me interested in the course with audial and visually enriched course content. 17 56.7 10 33.3 3 10 - - - - 4.47 .68 15. made it easier for me to follow the course content 15 50 13 43.3 2 6.7 - - - - 4.43 .63 16. affected my learning positively because of giving the opportunity to make up for the lessons I missed by myself. 16 53.3 14 46.7 - - - - - - 4.53 .51 17. was inefficient due to technical problems. - - 4 13.3 6 20 15 50 8 26. 7 2.00 .79 18. was more difficult than the methods used in the other courses. - - 4 13.3 6 20 15 50 5 16. 7 2.30 .91 International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 303 Table 4. Frequencies, percentages, mean scores and standard deviations of pre-service teachers’ perceptions about flipped learning Flipped Learning Strongly agree Agree Partly agree Disagree Strongly disagree M SD f % f % f % F % f % 19. supported planned and systematic work. 9 30 14 46.7 6 20 1 3.3 - - 4.03 .81 20. supported my independent learning skills. 7 23.3 21 70 2 6.7 - - - - 4.17 .53 21. increased my social interaction. 5 16.7 17 56.7 5 16.7 - - - - 4.10 .66 22. increased my research and exploration ability. 7 23.3 12 40 9 30 2 6.7 - - 3.80 .89 23. developed my self-learning skills. 12 40 14 46.7 3 10 1 3.3 - - 4.23 .77 24. took it longer to learn. - - 10 33.3 6 20 12 40 2 6.7 2.80 .99 25. made learning fun. 12 40 11 36.7 6 20 1 3.3 - - 4.13 .86 26. was adopted by me more than the other methods. 11 36.7 15 50 3 10 1 3.3 - - 4.20 .76 27. yielded more than the courses taught by other methods. 12 40 14 46.7 4 13.3 - - - - 4.27 .69 28. supported and facilitated my learning. 10 33.3 17 56.7 3 10 - - - - 4.23 .63 29. was distracting because there were many activities. 1 3.3 2 6.7 6 20 14 46.7 7 23.3 2.20 .99 30. attracted my attention much because of the use of different technologies. 9 30 10 33.3 8 26.7 3 10 - - 3.83 .99 31. provided faster learning. 7 23.3 17 56.7 5 16.7 1 3.3 - - 4.00 .74 32. was effective in practicing theoretical knowledge. 10 33.3 17 56.7 2 6.7 1 3.3 - - 4.20 .71 33. affected my learning positively 12 40 14 46.7 4 13.3 - - - - 4.27 .69 Özüdoğru&Aksu 304 * Items 19-33 were evaluated over 5. In terms of personal learning and contribution pre-service teachers agreed that flipped learning developed self-learning skills (86.7%), supported independent learning skills (93.3 %), planned and systematic work (76.7%) and increased research and exploration abilities (63.3%). Moreover, in terms of classroom environment perceptions pre-service teachers agreed that it increased social interaction (73.4%). On the other hand, pre-service teachers agreed that flipped learning took longer time to learn (33.3%) and was not distracting because of being so many activities (70%). 3.2. The results obtained from interviews on flipped learning In addition to these, the perceptions of pre-service teachers about the videos used in flipped learning was asked. The responses of pre-service teachers were categorized under seven sub-themes. They were 1) the perceptions of pre-service teachers about the voice and images of the videos, 2) the quantity of information in the videos, 3) factors effecting the desire to watch videos, 4) recurring watching of videos on learning, 5) appropriateness of time of sending the videos, 6) the obligation to watch the videos before class on pre-service teachers‟ learning and 7) the internet videos. 3.2.1. Voice and images of the videos According to the analysis of the interview process, all of the pre-service teachers interviewed generally had positive opinions about the videos. Majority of the pre-service teachers expressed their positive opinions about video lessons. They stated that the voices and images of the videos were clear and understandable. Also, pre-service teachers stated that videos made them interested in the course because of audial and visually enriched course content. There were background music in the videos. At the class, pre-service teachers explained that it was higher in one video and later its level was decreased by the instructor. Some of the responses about the videos used in flipped learning include: Mu (High Achiever): I think the videos were good. The videos were good in terms of listening and summarizing ... the quality of sound was good. Me (Medium Achiever): I could see everything thoroughly…sometimes there were schemas…the videos were good for me…and it's even nicer their being pictured and colored. Me (Low Achiever): Of course the sounds and images of the videos were understandable… In addition to these, pre-service teachers‟ suggestions to implement flipped learning effectively was asked. According to the perceptions of pre-service teachers, there might be animations in the videos, the instructor should appear in the foreground with gestures and facial expressions, and finally videos should end up with a quiz. Some of the suggestions of pre-service teachers include: because of involving group work. International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 305 Mu (High Achiever): Maybe if you had ended up the videos with test… if there were a 10 or 15 questions-test at the end of the videos, it would be great in terms of learning. Me (Low Achiever): You appear at the margin of the page but I wish you were in the foreground with your gestures and your facial expressions...I said this because maybe I am accustomed to learning this way. 3.2.2. The quantity of information in the videos Pre-service teachers expressed that the videos included sufficient information which helped them to learn the subject. Some of the responses about the quantity of information in the videos included: Fa (High Achiever): In my opinion, the content was pretty sufficient, also the questions related to the content were good. En (Medium Achiever): You split the videos ...If you set an eight-minute video, when we watched this eight-minute video, it was enough. I mean, it was very brief and plain than any other workbook. Me (Medium Achiever): There was enough information about the content… I think the information was enough for me since I like summary information...I could see everything thoroughly... Me (Low Achiever): There was enough information and even there was information that did not exist in some other books. 3.2.3. Factors affecting pre-service teachers‟ desire to watch videos The pre-service teachers stated the incentives as the visual components, the expressions of the instructor together with slides, desire to participate in class activities and discussions and the pop-up questions in the videos as the encouraging elements to watch videos. Some of the responses about the factors effecting pre-service teachers‟ desire to watch videos included: Bu (High Achiever): There were no elements to reduce my desire to watch videos, but I knew that when I did not watch the video, I would not be able to participate in the class activities when I came to class the next day, which encouraged me to watch videos... I listened to the videos since I wanted to talk in the class. Me (Medium Achiever): There were both questions and in the meantime, you were inserting different videos which were pointed in blue… I think they were quite enough and they were beneficial. Ha (Low Achiever): Since we raced in the lesson, I had so much fun… I watched the videos because I wanted to be always first in them. Me (Low Achiever): There were visual items. There were slides ... the expressions of the instructor together with slides encouraged me to watch videos. Furthermore, pre-service teachers explained the internet and the music behind the voice of the instructor as factors that reduced their desire to watch videos. Some of the Özüdoğru&Aksu 306 responses about the factors that reduced pre-service teachers‟ desire to watch videos included: Ra (High Achiever): We did not have powerful internet since we lived in KYK Dorm (Credit and Dormitories Institution - state dorm). Also, the internet lab in the university was not open the time we had time or we had different lessons at the time they were open. Hence, we spent much time to listen and send back to you. Fa (High Achiever): There was a problem only in terms of watching videos ... internet ... I waited 3 hours on the internet ... Edmodo did not open but the other sites opened ... The installation period of Edmodo took more times…I watched most of the homework videos at the state dorm but since I could not send them, because Edmodo was not installed, I came to the university and sent them here. Fa (Medium Achiever): The music in the videos made me drowsy…it was a little suppressing your voice... Me (Low Achiever): I would rather the videos were without music… there would not be any music at all... I wish only the voice of the instructor, and there would not be any sound out of it. 3.2.4. Recurring watching of videos on learning Pre-service teachers revealed their perceptions about recurring watching of videos on learning. Pre-service teachers‟ explanations showed that they would watch the videos whenever they felt the need which would contribute to learning. Also, pre-service teachers stated their having the opportunity to go back and watch some parts again affected their learning positively. Some of the responses for the recurring watching of videos include: Nu (High Achiever): The video courses was very good for me. I could stop it, I was taking my note, and then I was going back to beginning of the video when I needed. I was learning by watching the videos again. So, until the video was over, I had been watching it two or three times. Before the course, we were prepared. Also, you were explaining and conducting activities in class. By this way, learning was very permanent. In terms of learning, I benefitted more. Me (Medium Achiever): Naturally, when I did not understand something, I went back and studied again. I will watch all of the videos again before the final exam. Me (Low Achiever): I watched videos once because the videos were short and clear ...but I feel the need to watch them once more before the final exam. 3.2.5. The appropriateness of time to send the videos According to the perceptions of pre-service teachers, the time to send the videos were appropriate. Some of the responses about the time to send the videos included: International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 307 Ga (High Achiever): You were sending the videos in time, we were watching them before the course and taking our notes. Fa (High Achiever): It was appropriate, generally, you sent videos on Friday, and we had at least the weekend to study. Me (Medium Achiever): You sent them in two or three day after the course... there was time to summarize ... On the other hand, one of the high achiever pre-service teacher suggested that videos should be sent a little earlier. The suggestions of this pre-service teacher include: Ga (High Achiever): You sent the videos generally on Friday... if you sent the videos a little earlier, we would watch them earlier and submit you earlier. 3.2.6. The obligation to watch the videos before class on pre-service teachers‟ learning Pre-service teachers explained their perceptions about the obligation of watching videos before class on their learning. They indicated that if the watching of videos were not obligatory, they would not feel the responsibility of watching it. Some of the responses of pre-service teachers include: Ra (High Achiever): If watching the videos were not compulsory, we might not watch them regularly. We might say ourselves that I didn’t want to watch them today, and ask ourselves that whether we had to watch them every week. When students felt tired...they could say that “I am tired and I will not study today. Me (Middle Achiever): This obligation is a good thing in terms of learning, but at first this obligation was not good for me because students at first could not think that some obligations would be beneficial for them later hence they should learn it or do the tasks. Me (Low Achiever): I would not have felt any responsibility if you had not forced me… I would watch videos every two or three weeks…It was better in this way, I learned without the content accumulated. Ha (Low Achiever): We watched the videos you sent to us, we summarized and took notes, then we sent them back to you... At first, I thought that this process as a bit pointless, but after a while I understood that we would not study to the course if we did not send the summaries or assignments to you every week ... the videos would stay as you sent. 3.2.7. Perceptions about the use of internet videos Pre-service teachers stated that they did not watch internet videos related to the subject to learn the lesson. They stated that the videos sent by the instructor were more understandable and concise to learn the content than the internet videos. Some of the responses of pre-service teachers about the videos used in flipped learning included: Özüdoğru&Aksu 308 Bu (High Achiever): I watched the video on the internet, they were around two hours... they explained the topic in very long times…but with the videos you sent, we could learn the topic in a very short time with clear outline… Me (Medium Achiever): I did not need it because they were too long and I was tired of watching them. Hence, after a while, I got distracted and I never watched the internet videos for this course. 4. Discussion According to the results of the study, pre-service teachers agreed that flipped learning supported and facilitated learning, was effective in order to practice theoretical knowledge and contributed to personal development and self-enhancement. In the current study, pre- service teachers watched lecture videos and while watching them, pre-service teachers had the opportunity of turning back and watching some parts again whenever they needed, taking notes, writing questions to be asked in the class which might have affected the learning and perceptions of pre-service teachers positively in the flipped learning group. Similar to the current study, in the study conducted by Enfield (2013), it was reported that majority of students took notes and found it helpful to learn the course. Moreover, in the study conducted by B. Aydin (2016), pre-service teachers explained that they could learn the content according to their own learning speed and they could watch some videos again when they did not understand. Similarly, in the study conducted by Ceylaner (2016), ninth grade students in English course also stated that when they forgot some rules, they had the chance to turn back to the related video and watch the subject again. In this way, according to them, learning was deep and more permanent. Also, Day and Foley (2006) stated that according to Dale‟s Cone of Learning (1969), videos fall in the middle of the cone in terms of retention and they explained active learning experiences and participating in hands-on learning activities increase the retention of learning as it is in the flipped learning. In the current study, pre-service teachers who were in the experimental group were presented with course materials in several different formats like slides, the expressions of the instructor together with slides, sample videos from You Tube, and the pop-up questions, which might be another reason for the pre-service teachers‟ positive perceptions of flipped learning. It can be said that visual and audial materials might have increased participation in class activities by making the activities fun increased the involvement of pre-service teachers with different learning styles and preferences, thus, supported learning. Similarly, Lage et al. (2000) revealed that flipped learning support all students of all learning styles to use methods that best match for them. Umutlu (2016) prepared different video modalities by taking the participants‟ learning styles which was stated as one of the important variable while examining the effects of the flipped learning on achievement in English. In addition to these, in the current study, pre-service teachers stated that learning with videos was fun. Furthermore, Kahoot and Socrative softwares, might have increased the fun in the face to face part of the course which in turn affected their perceptions positively. Similarly, in the study conducted by Boyraz (2014), it was found that according to English preparatory class students, learning through videos was fun and they preferred studying English via videos to studying English by underlining important points from course books. This may also be explained as pre-service teachers perceived the materials used in the flipped learning as attractive. Moreover, in the current study, it was determined that practicing the theoretical knowledge might be another reason for the pre-service teachers‟ positive perceptions of flipped learning. International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 309 They prepared lesson plans, concepts maps participated in micro-teaching activities in order to practice theoretical knowledge. Hence, they might have perceived that they learned more and their learning was permanent. Similarly, in the study conducted by Turan (2015), pre- service teachers also stated that flipped learning was based on practice and increased the permanence of the learning. In this way, they did not memorize the content but they actually learned the content and learning did not take longer time. Hence, it can be suggested that when flipped learning is integrated with group tasks, discussions and active student presentations, it increase the involvement and learning of students without taking much time, as it was also found in the current study. Furthermore, similar to the current study, in the study conducted by Mason et al. (2013), students explained that they spent significantly fewer hours per week studying outside of the classroom than the students who took the course in the traditional classroom. The reason for this might have stemmed from the fact that pre-service teachers were provided with brief and interactive videos which took less time to watch and summarize the content. In this way, they could focus on the important points while learning instead of reading many pages and did not know which parts to focus on. Hence, it is suggested that videos should be brief and concise in order to not to be students‟ bored and should include interactive pop-up questions. In addition to these, learning with videos provided pre-service teachers with access to the course content independent of the space, which might be another reason for the positive perception of pre-service teachers about flipped learning. In this way, they had the opportunity of making up the courses by themselves which is in line with the previous literature (Alsancak-Sırakaya, 2015; Bergman & Sams, 2012; Guc, 2017). In this way, it might be said that flipped learning contributed to individual learning abilities, supported independent learning skills and pre-service teachers had more control over their learning, as stated by pre-service teachers in the student questionnaire. These are some of the important properties of flipped learning and pre-service teachers are positive about flipped learning due to these properties. In the current study, pre-service teachers stated that learning with videos was easier. In this study, around 21 minutes videos were divided into brief parts, the content was shorter and plain than any other course book, which may be an important reason for the fact that flipped learning affected their learning positively. In the literature, there are many studies supporting this finding (Bergmann & Sams, 2012; Caudill, 2014; Enfield, 2013; Turan, 2015). Even in the study of Ceylaner (2016) it was found that some ninth grade students checked the duration of videos before watching them and watched the videos by skipping some parts which were more than five minutes. Hence, it is suggested that as in the current study, videos should be divided into two or three parts to be managed by pre-service teachers easily. Moreover, the reason for the positive perceptions of pre-service teachers about flipped learning might have stemmed from video courses prepared by instructor than internet videos. According to content analysis, pre-service teachers stated that they preferred watching video courses prepared by instructor than watching the internet videos about the same topic. Similarly, in the study conducted by Turan (2015), pre-service teachers in the Early Childhood Education department explained their preference to watch the videos prepared by the instructor instead of ready-made internet videos. Dove & Dove (2017) found that flipped learning with teacher-created videos decreased elementary education pre-service teachers‟ mathematics anxieties and increased confidence in mathematics more than did instruction that incorporated in-class lectures or third-party videos. One of the reasons for this might be that most of the videos prepared for Principles and Method of Instruction course are prepared Özüdoğru&Aksu 310 by special institutions to prepare pre-service teachers for KPSS examination (Public Personnel Selection Examination), and they are over one hour and include full of terms and extra topics that pre-service teachers are not familiar with. Day & Foley (2006) explained that while preparing web/video lectures, professional production quality is not necessary and evidence from focus groups suggested that informal recordings were found more enjoyable. Hence, it can be said that brief, clear and concise videos prepared by the instructor might have been perceived by pre-service teachers as more understandable to learn the content than the internet videos, which might be the reason for the positive effect of flipped learning according to perceptions of pre-service teachers. In addition to these, pre-service teachers perceived that flipped learning contributed to student interactivity and collaboration when compared to traditional courses. The pre-service teachers who took part in flipped group expressed an increase in collaboration with their friends and instructors both in the classroom and out of the classroom by using technology, which is in line with previous literature (Baker, 2000; Munir, Baroutiana, Younga, & Carter, 2018). In this way, pre-service teachers cooperated with others when completing assignments and could learn from each other. Similarly, in the study conducted by Ceylaner (2016), ninth grade students stated that they shared a lot with their classmates because of being involved in group tasks. They stated that they solved their problems together and even shared videos via Bluetooth with their classmates to be watched by the peers who did not have internet connection. In the study conducted by Guc (2017), among 13 secondary school students, 11 of them indicated positive opinions about flipped learning on peer learning. Moreover, in the current study, pre-service teachers explained that since everyone would get the same grade from the group work, they communicated with those people even if they were not close friends. Also, they stated that because it was a joint work, all of them put effort. They worked together to achieve their group goals. In other words, their communication with peers during class affected learning positively. Furthermore, increased faculty-student interaction might be one of the reasons for the positive perceptions of flipped learning group which is in line with the previous literature (Alsancak-Sırakaya, 2015; Bergmann & Sams, 2012; Berrett, 2012; Ceylaner, 2016; Findlay- Thompson & Mombourquette, 2014; Lage et al., 2000; McLaughlin et al., 2013; Milman, 2012; Ziegelmeier & Topaz, 2015). In the current study, during in-class activities the instructor was able to monitor the flipped group‟s performance and comprehension and when a misunderstanding or any confusion was noticed, they were cleared up immediately. Hence, the instructor has many opportunities to give feedback, which eases the learning process and creates a sense of involvement and acceptance by creating a positive classroom environment. Also, in the current study, pre-service teachers interacted with the instructor out of the class by using social media and the instructor could help them anytime they needed. In the study of Findlay-Thompson and Mombourquette (2014), Introduction to Business Administration students reported that talking to the professor in and out of the class was easier in the flipped learning, which was also the case in the current study. In addition to these, similar to the current study, den Brok, Brekelmans, & Wubbels (2004) indicated that teacher proximity was important for pleasure in courses. Ghaith (2002) showed that the more the learners received academic and personal teacher support rather than peer support, the more they perceived that they could better adjust socially and psychologically at school. Finally, pre-service teachers mentioned the weaknesses of flipped learning which is in line with the literature (B. Aydın 2016; G. Aydın 2016; Caudill, 2014; Enfield, 2013; Guc, 2017; Milman, 2012; Yavuz, 2016). In the current study, the lack of fast internet connection and the music behind the voice of the instructor were stated as some factors affecting their desire to watch videos. These kinds of technological issues might have affected the satisfaction of pre- International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 311 service teachers from flipped learning. Similarly, Caudill (2014) stated that the downsides of using technology in a classroom setting were the program crashing, not working applications, and uninteresting or too long videos. In the studies, conducted by Enfield (2013) and Milman (2012), it was displayed that some technical issues like accessing, streaming and downloading the videos annoyed many students and affected their learning negatively. In the study conducted by Guc (2017), for some secondary school mathematics students it was an important issue to open videos because of viruses or other reasons. In the present study, since the videos were divided into parts, the file size was not large. Hence, watching them from their phones or jumping forward or backward in the video did not create a problem. However, because of slow internet speed in the student dormitories, they had problems to open Edmodo. This study revealed the perceptions of pre-service teachers related to flipped learning. The findings obtained from this study can be taken into consideration by the practitioners to effectively design flipped learning environments. As it was stated by Butt (2014), there is always going to be room for improvement. Whether these improvements take place in lecture videos, learning activities, organization or students‟ learning experiences, instructors should be open to making changes whenever it is necessary according to grades of their students and the courses they taught. Özüdoğru&Aksu 312 References Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1-14. doi: 10.1080/07294360.2014.934336. Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1-14. doi: 10.1080/07294360.2014.934336. Alsancak-Sırakaya, D. (2015). Tersyüz sınıf modelinin akademik basari, öz-yönetimli öğrenme hazırbulunuşluğu ve motivasyon üzerine etkisi [The effect of flipped classroom model on academic achievement, self-directed learning readiness and motivation]. (Doctoral dissertation). Gazi University, Ankara. Aydin, B. (2016). Ters yüz sınıf modelinin akademik başarı, ödev/görev stres düzeyi ve öğrenme transferi üzerindeki etkisi [The effects of flipped classroom model on academic achievement, homework/task stress level and transfer of learning]. (Master‟s thesis). Süleyman Demirel University, Isparta. Aydin, G. (2016). Ters yüz sinif modelinin üniversite öğrencilerinin programlamaya yönelik tutum, öz-yeterlik algısı ve başarılarına etkisinin incelenmesi [The investigation of the effect of flipped classroom model on undergraduate students' attitude, self-efficacy and academic achievement towards programming]. (Master‟s thesis). Dokuz Eylül University, İzmir. Baker, J. W. (2000). The classroom flip: Using web course management tools to become the guide by the side. In J. A. Chambers (Ed.), Selected papers from the 11 th International Conference on College Teaching and Learning (pp. 9-17). Jacksonville, FL: Florida Community College at Jacksonville. Retrieved from http://www.classroomflip.com/files/classroom_flip_baker_2000.pdf Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. Alexandria, VA: International Society for Technology in Education. Berrett, D. (2012). How flipping the classroom can improve the traditional lecture. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/How Flipping-the-Classroom/130857/ Bishop, J. L., & Verleger, M. A. (2013, June). The flipped classroom: A survey of the research. Proceedings of American Society for Engineering Education (ASEE) National Conference. Article retrieved from http://www.asee.org/public/conferences/20/ papers/6219/view Bormann, J. (2014). Affordances of flipped learning and its effects on student engagement and achievement (Doctoral dissertation). University of Northern Iowa. Chicago. Retrieved from https://tinyurl.com/y8uxxbjn Boyraz, S. (2014). İngilizce öğretiminde tersine eğitim uygulamasının değerlendirilmesi [Evaluating flipped classroom/education method in English teaching]. (Master‟s thesis). Afyon Kocatepe University, Afyon. Butt, A. (2014). Student views on the use of a flipped classroom approach: Evidence from Australia. Business Education & Accreditation, 6(1), 33-43. Retrieved from http://search.proquest.com/docview/1446438932?accountid=14691 International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 313 Caudill, N. V. (2014). Pre-service teachers’ perceptions of a flipped classroom: A study of undergraduates enrolled in an applied child development course (Master‟s thesis). Graduate Faculty of North Carolina State University, Raleigh, North Carolina. Retrieved from https://tinyurl.com/ybygdadr Ceylaner, S. (2016). Dokuzuncu sınıf İngilizce öğretiminde ters yüz sınıf yönteminin öğrencilerin öz yönetimli öğrenmeye hazırbulunuşluklarına ve İngilizce dersine yönelik tutumlarına etkisi [Effects of flipped classroom on students’ self- directed learning readiness and attitudes towards English lesson in 9 th grade English language teaching]. (Master‟s thesis). Mersin University, Mersin. Cibik, B. (2017). The effects of flipped classroom model on learner autonomy. [Tersyüz eğitim modelinin öğrenen özerkliği üzerine etkileri]. (Master’s thesis). Muğla Sıtkı Koçman University, Muğla. Clark, K. R. (2013). Examining the effects of the flipped model of instruction on student engagement and performance in the secondary mathematics classroom: An action research study (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3592584). Cohen, L., Manion, L. & Morrison, K. (2007). Research methods in education (6 th Ed.). New York: Routledge. Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Boston: Pearson. Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educational Technology Research and Development, 61(4), 563-580. doi: 10.1007/s11423-013-9305-6 Day, J. A. & Foley, J. D. (2006). Evaluating a web lecture intervention in a human computer interaction course. IEEE Transactions on Education, 49(4), 420-431. doi: 10.1109/TE.2006.879792 den Brok, P., Brekelmans, M., & Wubbels, T. (2004). Interpersonal teacher behavior and student outcomes. School Effectiveness and School Improvement, 15(3-4), 407-442. doi: 10.1080/09243450512331383262. Dove, A., & Dove, E. (2017). Flipping preservice elementary teachers‟ mathematics anxieties. Contemporary Issues in Technology and Teacher Education, 17(3), 312-335. Retrieved from https://citejournal.s3.amazonaws.com/wp content/uploads/v17i3math1.pdf Enfield, J. (2013). Looking at the impact of the flipped classroom model of instruction on undergraduate multimedia students at CSUN. Techtrends: Linking Research & Practice to Improve Learning, 57(6), 14-27. doi:10.1007/s11528-013-0698 Field, A. P. (2009). Discovering statistics using spss. (3 th Ed.). London: SAGE Publications. Findlay-Thompson, S., & Mombourquette, P. (2014). Evaluation of a flipped classroom in an undergraduate business course. Business Education & Accreditation, 6(1), 63-71. Retrieved from https://tinyurl.com/y95c8898 Foertsch, J., Moses, G., Strikwerda, J., & Litzkow, M. (2002). Reversing the lecture/homework paradigm using eTEACH web-based streaming video software. Özüdoğru&Aksu 314 Journal of Engineering Education, 91(3), 267-274. doi: 10.1002/j.2168 9830.2002.tb00703.x Fraenkel, J. R. & Wallen, N. E. (2009). How to design and evaluate research in education (7 th ed.). New York: McGraw Hill. Fulton, K. P. (2012). 10 reasons to flip. Phi Delta Kappan, 94(2), 20-24. Retrieved from https://doi.org/10.1177/003172171209400205 Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction (7 th ed.). Boston: Allyn-Bacon. Gaughan, J. E. (2014). The flipped classroom in world history. History Teacher, 47(2), 221-244. Retrieved from https://tinyurl.com/y7em57aq Ghaith, G. M. (2002). The relationship between cooperative learning, perception of social support, and academic achievement. System 30, 263–273. doi: 10.1016/S0346 251X(02)00014-3 Guc, F. (2017). Rasyonel sayılar ve rasyonel sayılarda işlemler konusunda ters-yüz sınıf uygulamasının etkileri [The effect of the flipped classroom practice on the rational numbers and operations with rational numbers]. (Master‟s thesis). Amasya University, Amasya. Hamdan, N., McKnight, P., McKnight, K., & Arfstrom, K (2013). A review of flipped learning. Flipped Learning Network. Retrieved from https://tinyurl.com/y9slgecn Kecskemety, K. M., & Morin, B. (2014, June). Student perceptions of inverted classroom benefits in a first-year engineering course. Paper presented at 121 st ASEE Annual Conference & Exposition, Indianapolis. Retrieved from https://peer.asee.org/student perceptions-of-inverted-classroom-benefits-in-a first- year-engineering-course Kurt, G. (2017). Implementing the flipped classroom in teacher education: Evidence from Turkey. Educational Technology & Society, 20(1), 211–221. Retrieved from https://www.jstor.org/stable/jeductechsoci.20.1.211 Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. Journal of Economic Education, 31(1), 30-43. Retrieved from http://dx.doi.org/10.1080/00220480009596759 Lee, J., Lim, C., & Kim, H. (2016). Development of an instructional design model for flipped learning in higher education. Education Tech Research Dev, 65, 427-453. doi: 10.1007/s11423-016-9502-1 Love, B., Hodge, A., Grandgenett, N., & Swift. A. W. (2014). Student learning and perceptions in a flipped linear algebra course. International Journal of Mathematical Education in Science and Technology. 45(3), 317–324. doi: 10.1080/0020739X.2013.822582 Mason, G. S., Shuman, T. R., & Cook, K. E. (2013). Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course. IEEE Transactions on Education, 56(4), 430-435. doi: 10.1109/TE.2013.2249066. McLaughlin, J. E., Griffin, L. M., Esserman, D. A., Davidson, C. A., Glatt, D. M., Roth, M. T., Gharkholonarehe, N., & Mumper, R. J. (2013). Pharmacy student engagement, International Online Journal of Education and Teaching (IOJET) 2019, 6(2), 295-316 315 performance, and perception in a flipped satellite classroom. American Journal of Pharmaceutical Education, 77(9), 1-8. doi: 10.5688/ajpe779196. Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: A methods sourcebook (3 rd Ed.). United States: SAGE Publications. Milman, N. (2012). The flipped classroom strategy: What is it and how can it best be used? Distance Learning, 9(3), 85-87. Retrieved from https://search.proquest.com/openview/616e91b3df376d82fd5d30c598c665f3 1?pq origsite=gscholar&cbl=29704 Missildine, K., Fountain, R., Summers, L., & Gosselin, K. (2013). Flipping the classroom to improve student performance and satisfaction. Journal of Nursing Education, 52(10), 597-599. doi: 10.3928/01484834-20130919-03 Morin, B., Kecskemety, K. M., Harper, K. A., & Clingan, P. A. (2013, June 23-26). The inverted classroom in a first-year engineering course. Paper presented at 120 th ASEE Annual Conference & Exposition. Retrieved from https://www.asee.org/public/conferences/20/papers/7230/view Munir, M. T., Baroutian, S., Young, B. R., & Carter, S. (2018). Flipped classroom with cooperative learning as a cornerstone. Education for Chemical Engineers 23, 25–33. Retrieved from https://doi.org/10.1016/j.ece.2018.05.001 Murphree, D. S. (2014). Writing wasn't really stressed, accurate historical analysis was stressed: Student perceptions of in-class writing in the inverted, general education, university history survey course. The History Teacher, 47(2), 209-219. Retrieved from https://tinyurl.com/ya6wvcvy O‟Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. Internet and Higher Education, 25, 85-95. Retrieved from http://dx.doi.org/10.1016/j.iheduc.2015.02.002 Phillips, C. R., & Trainor, J. E. (2014). Milennial students and the flipped classroom. Proceedings of the ASBBS Annual Conference, 21(1) 519-530. Retrieved from http://asbbs.org/files/ASBBS2014/PDF/P/Phillips_Trainor(P519-530).pdf Reinhardt, J. Y. (2014). Improving classroom practice through collaborative inquiry: A case of flipped learning. (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3624225). Sams, A., & Bergmann, J. (2013). Flip your students' learning. Educational Leadership, 70(6), 16-20. Retrieved from https://tinyurl.com/bssk349 Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (6 th Ed.). Boston: Pearson Education Inc. Talbert, R. (2012). Inverted classroom. Colleagues, 9(1), 1-3. Article retrieved from http://scholarworks.gvsu.edu/colleagues/vol9/iss1/7. Talbert, R. (2014). Inverting the linear algebra classroom. Primus, 24(5), 361-374. doi: 10.1080/10511970.2014.883457 Talley, C., & Scherer, S. (2013). The enhanced flipped classroom: increasing academic performance with student-recorded lectures and practice testing in a flipped stem course. Journal of Negro Education, 82(3), 339-347. doi: 10.7709/jnegroeducation.82.3.0339 Özüdoğru&Aksu 316 Touchton, M. (2015). Flipping the classroom and student performance in advanced statistics: Evidence from a quasi-experiment. Journal of Political Science Education, 11(1), 28- 44. doi: 10.1080/15512169.2014.985105 Tune, J. D., Sturek, M., & Basile, D. P. (2013). Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Adv Physiol Educ, 37(4), 316-320. doi:10.1152/advan.00091.2013 Turan, Z. (2015). Ters yüz sınıf yönteminin değerlendirilmesi ve akademik başarı, bilişsel yük ve motivasyona etkisinin incelenmesi [The evaluation of flipped classroom method and examination of its effects on academic achievement, cognitive load and motivation]. (Doctoral dissertation). Atatürk University, Erzurum. Turgut, M. F., & Baykul, Y. (2011). Eğitimde ölçme ve değerlendirme (3 rd Ed.). Ankara: Pegem Akademi. Umutlu, D. (2016). Effects of different video modalities on writing achievement in flipped English classes (Master‟s thesis). Boğaziçi University, İstanbul. Wilson, S. G. (2013). The flipped class: A method to address the challenges of an undergraduate statistics course. Teaching of Psychology. 40(3), 193-199. doi:10.1177/0098628313487461 Yavuz, M. (2016). An investigation into the effects of flipped classroom applications on the academic success and experiences of the students at secondary school [Ortaöğretim düzeyinde ters yüz sınıf uygulamalarının akademik başarı üzerine etkisi ve öğrenci deneyimlerinin incelenmesi]. ( Unpublished Thesis). Atatürk University, Erzurum. Zappe, S., Leicht, R., Messner, J., Litzinger, T., & Lee, H. (2009). Flipping the classroom to explore active learning in a large undergraduate course. Proceedings of the 2009 American Society for Engineering Education Annual Conference and Exhibition. Retrieved from https://tinyurl.com/y93n7r3j Ziegelmeier, L. B., & Topaz, C. M. (2015). Flipped calculus: A study of student performance and perceptions. Primus, 25(9-10), 847-860. doi: 10.1080/10511970.2015.1031305.