Australasian Journal of Educational Technology, 2020, 36(4). 27 Pre-service teachers’ achievement and perceptions of the classroom environment in flipped learning and traditional instruction classes Melike Özüdoğru Manisa Celal Bayar University, Turkey Meral Aksu Middle East Technical University, Turkey The aim of this study was to investigate whether flipped learning affects pre-service teachers’ achievement and perceptions related to the classroom environment. This experimental study was conducted in the fall semester of 2017–2018 for 11 weeks at a state university in Turkey and included a total of 56 pre-service teachers. An achievement test and a scale were implemented to collect the data. The analyses through descriptive and inferential statistical analysis techniques showed that flipped learning group obtained significantly higher achievement test scores and final grades than the traditional instruction group. On the other hand, being in the flipped learning group or in the traditional group had no significant effect on the pre-service teachers’ perceptions of the classroom environment. Implications for practice or policy: • University administrators may organise in-service training programs for instructors on computer applications, software, and games as implementing them in courses may increase learner satisfaction and contribute to a more positive classroom environment. • Instructors need to consider the duration of lecture videos as videos should not be too long. • Instructors need to consider the length and content of lecture videos – students are more likely to come to class prepared if videos are kept to a manageable length and the content is “chunked”. Keywords: flipped learning, classroom environment perceptions, pre-service teacher education, student achievement, principles and methods of instruction course Introduction We live in the technology era when different groups such as politicians, administrators, parents and concerned people or groups want to have clear and straightforward evidence that students are truly learning in higher education institutions (Berrett, 2012). The actions of higher education institutions have been inspected carefully for the efficient use of class time and assessment techniques (Ziegelmeier &Topaz, 2015). Instructors are seen not as the sage on the stage but as the guide by the side (Baker, 2000). Students are no longer expected to be passive receivers of knowledge who just listen to the teacher or read the information from books (Touchton, 2015). Moreover, students have gained flexibility in terms of the place and time for their learning, they have rich learning contexts and have benefitted from distance education opportunities with the new technological developments. Furthermore, according to the Horizon 2014 report, education paradigms are shifting and instruction includes the use of technology through online and blended learning by benefitting from the principles of collaborative learning (Johnson, Adams Becker, Estrada, & Freeman, 2014). For this reason, various educational researchers have proposed flipped learning, an active teaching-learning approach (Zappe, Leicht, Messner, Litzinger, & Lee, 2009; Ziegelmeier & Topaz, 2015). According to Lage, Platt, and Treglia (2000) flipped learning means that “events that have traditionally taken place inside the classroom now take place outside the classroom and vice versa” (p. 32). In other words, the basic idea of flipped learning is that instead of allocating time to present a concept through lecturing, instructors can use this valuable and limited class time for more involving activities such as class discussions, peer-supported learning and group work. Bishop and Verleger (2013) stated that flipped learning consists of two parts: interactive group learning activities which were conducted in class and direct Australasian Journal of Educational Technology, 2020, 36(4). 28 computer-based individual instruction which is conducted outside class. It can be said that the basic principle of flipped learning is that students begin to learn the course subject by themselves and complete pre-class activities by benefitting from computer technologies in order to take the advantage of active in- class tasks efficiently (Abeysekera & Dawson, 2015; Bergmann & Sams, 2012; Mason, Shuman, & Cook, 2013; Phillips & Trainor, 2014). Flipped learning uses class time effectively by integrating the necessary technology and providing collaborative learning opportunities (Caudill, 2014; Findlay-Thompson & Mombourquette, 2014; Millard, 2012). When students share information in collaborative activities, both communication opportunities among peers and their enjoyment of learning increase (Foertsch, Moses, Strikwerda, & Litzkow, 2002; Talbert, 2012; Touchton, 2015; Walker, 2003). In the literature, most of the studies investigated the effect of flipped learning on the achievements of students along with other cognitive and affective variables. Researchers have determined that flipped learning improved student learning and achievement in a wide variety of courses including statistics (Touchton, 2015; Wilson, 2013); physiological psychology (Talley & Scherer, 2013); cardiovascular, respiratory and renal physiology (Tune, Sturek, & Basile, 2013); introductory Excel (Davies, Dean, & Ball, 2013); mathematics (Fulton, 2012); applied linear algebra (Love, Hodge, Grandgenett, & Swift, 2014); mechanical engineering program (Mason et al., 2013); history (Murphree, 2014); human–computer interaction (Day & Foley, 2006) and chemistry (Talbert, 2014). On the other hand, some of the studies have indicated insignificant differences in student learning (“Introduction to Business Administration”, Findlay- Thompson & Mombourquette, 2014; multivariable calculus, Ziegelmeier & Topaz, 2015). Also, Clark (2013) indicated that there was no significant difference between the performance of ninth grade students who were in the flipped learning group and those who were in the traditional group. Schwab (1969) stated that the curriculum should rely more on practical methods and principles that can be applied in real classroom environments than on theory. Schwab (1969) stressed the importance of the learning environment and stated that for effective instruction the properties of the classroom environment should be taken into account. These concepts and the framework of this study are shown in Figure 1. Figure 1. The framework of the study (adopted from Strayer (2007, p. 16) According to Fraser (1986, p. l), the word “environment” is defined as “shared perceptions of students and teachers in that environment”. Similarly, Dorman (2002, p. 2) defined the classroom environment as “the atmosphere, ambience, tone or climate that pervade the particular setting”. According to them, effective instruction may not be achieved if the dynamics of classroom environments are not taken into consideration. Australasian Journal of Educational Technology, 2020, 36(4). 29 Previous researchers have studied the classroom environment in terms of the physical properties of the environment, such as furniture, teaching materials, lighting, and the seating arrangements. But over the last few decades, the psychosocial aspect of the classroom environment has come to be thought more important (Fraser, 1986; LaRocque, 2008). The classroom environment involves psychosocial aspects such as satisfaction, cooperation, student involvement, task orientation, student cohesiveness, and difficulty (Fraser, 1980; Fraser, Giddings, & McRobbie, 1992; Fraser & Treagust, 1986; Rentoul & Fraser, 1980; Taylor, Fraser, & Fisher, 1997; Trickett & Moos, 1973). The satisfaction property of the classroom environment is about the students’ enjoyment of class work. Cooperation is about students working together rather than being in competition during learning tasks. Involvement is about the participation of students in activities and discussions, interest in the course, and the undertaking of additional research. In addition to these, when task orientation is taken into consideration, it is important for students to follow planned activities and continue working on the course content. Student cohesiveness is about the intimacy of relationships and related to students knowing, helping, and supporting each other. Difficulty refers to whether students find work difficult or not. Teacher support is related to the extent that teachers help, befriend, trust, and show interest in the students. Studies have indicated the importance of the positive psychosocial properties of the classroom environment for cognitive and affective outcomes of students, such as achievement (Chionh & Fraser, 2009; Roth 1998), attitude (Goh & Fraser, 2000), autonomy and student-centredness (Roth 1998), engagement (Dotterer & Lowe, 2011), motivation (Arısoy, 2007), satisfaction (Fraser & Treagust, 1986; Zandvliet, 1999), self- esteem (Chionh & Fraser, 2009), academic self-efficacy (Dorman, Adams, & Ferguson, 2003), self- regulation (Velayutham & Aldridge, 2013), by implementing different classroom environment instruments in different countries and grade levels (Fraser, 1998). In addition to these, the results of meta-analysis studies have shown that the dimensions of cohesiveness, satisfaction, task difficulty, formality, goal direction, democracy and the material environment are positively related to learning outcomes. On the other hand, friction, cliqueness, apathy, and disorganisation are negatively related to learning (Haertel, Walberg, & Haertel, 1981; O’Reilly, 1975). While teacher supportiveness, involvement, and satisfaction with the course were found as significant predictors of class participation in Turkey (Atbas, 2004), higher exam scores in classrooms were found in classrooms where students perceived more student cohesiveness in Singapore (Chionh & Fraser, 2009). Moreover, classroom environment perceptions were found to vary with certain variables such as gender, grade level, type of school, teacher qualification and class size (Kim, Fisher, & Fraser, 2000; LaRocque, 2008; Majeed, Fraser, & Aldridge, 2002). In the study conducted by Lee, Lee, and Wong (2003), primary school students tended to have significantly higher levels of collaboration, teacher support and involvement in the actual classroom environment than those of secondary schools in Hong Kong. On the other hand, in the western part of the United States of America, LaRocque (2008) found that students with lower grades perceived their classroom as more competitive and less cohesive than the students with higher grades, and as students’ grade levels increased, they became less satisfied with their classroom environment. These findings indicate that students’ perceptions differ according to student characteristics and inform teachers about the importance of structuring their classroom environments. Significance of the study and research questions Although previous research has suggested that student learning is better in a flipped learning environment compared to traditional classrooms, the issue is still open to debate. In this regard, we considered that the examination of the effect of flipped learning on pre-service teachers’ achievement may contribute both to the literature and studies related to pre-service teacher training, since the contribution of flipped learning on pre-service teachers’ achievement is unclear. The results of many studies revealed that students and teachers preferred a more optimal classroom environment than the actual classroom environment (Fraser & Treagust, 1986). However, teachers viewed their classroom environment more positively than their students (Chien, 2007; Zandvliet, 1999). In the study of McRobbie, Roth, and Lucus (1997), it was revealed that teachers did not know the diverse properties of the classroom environment and that the constraints of classroom environments were related to student learning. Hence, it was thought that this study would provide important results for the Australasian Journal of Educational Technology, 2020, 36(4). 30 psychosocial aspect of classrooms promoting or inhibiting learning in pre-service teacher training. Also, we considered that this experimental study would be of great importance because it is one of the first to investigate the classroom environment in both flipped and traditional classes with the aim of increasing the effectiveness of teacher training. This study was conducted in the “Principles and Methods of Instruction” course with the use of a flipped learning approach. Since traditional classrooms provide direct instruction in class and mostly expect students to practise what they have learned later in their future professions without applying it during training, which is a problematic situation for pre-service teachers, it was thought that teaching this course by implementing flipped learning may create an opportunity for pre-service teachers not only to observe and experience this new pedagogical approach at firsthand but also to deepen their understanding of the course content by presenting the knowledge outside of the class time. The aim of this study was to produce course videos and other materials to implement in the “Principles and Methods of Instruction” course and investigate whether flipped learning affected pre-service teachers’ achievement and perceptions related to the classroom environment. Based on the main purpose of the study, the following research questions were proposed: (1) Do flipped learning and traditional instruction groups differ in their achievement test scores and final grades? (2) Do flipped learning and traditional instruction groups differ significantly in their perceptions of the classroom environment (satisfaction, cooperation, involvement, task orientation, student cohesiveness, difficulty)? Method Research design This study employed quasi-experimental research design in which traditional instruction was implemented in the control group and instruction designed according to the principles of flipped learning was implemented in the experimental group. Educational context The study was conducted in the fall semester of 2017–2018 for 11 weeks at Manisa Celal Bayar University, which is a state university located in the Aegean Region in Turkey. The university offers over 60 programs at undergraduate and postgraduate levels. Nearly 3000 people (1333 administrative and 1649 academic staff including 55 foreigners working on contract) are involved in the administration of the university. The university has 55,000 students and is participating in the Bologna process since it aims to achieve better standards in all areas of education and supports educational programs around the world (Manisa Celal Bayar University, 2019). The Faculty of Education in which the study was conducted has six departments and approximately 1900 pre-service teachers. In the faculty, pre-service teachers are trained to obtain the scientific knowledge and skills needed in their profession. Hence, all courses are carried out in accordance with the framework program set by the Higher Education Council of Turkey (Yüksek Öğretim Kurumu, 2007). The duration of the training is determined as eight semesters in four academic years. Course information The study was conducted in a compulsory second-year educational course “Principles and Methods of Instruction”, which is taught three hours a week in all faculties of education in Turkey. The course aims to equip pre-service teachers with the knowledge of basic concepts related to education and instruction, learning and teaching, the main learning and teaching approaches, methods and techniques. Also, the course objectives are to have pre-service teachers implement different teaching strategies, methods; comprehend and put principles of planned instruction into practice; use suitable teaching materials and tools; become aware of teachers’ duties and responsibilities; and to comprehend the relationship among these with teacher Australasian Journal of Educational Technology, 2020, 36(4). 31 qualifications in accordance with the frameworks set by the Higher Education Council (Yüksek Öğretim Kurumu, 2007). Hence, the content of the course includes the following areas: basic concepts related to teaching, learning and education, teaching theories and models (e.g., Gagne’s theory on learning and instruction, Mastery learning, Keller’s individualised instruction, Carroll’s model of school learning, Bloom’s mastery learning, constructivism, multiple intelligence theory), teaching strategies, methods of teaching and different teaching techniques. Participants of the study In this study, out of three classes of the Elementary Education – Classroom Teaching Department, one class was chosen as the experimental group and one section was chosen as the control group randomly. Pre- service teachers were not informed about the instructional method to be used before they enrolled in the course. There were 56 pre-service teachers in total. While 30 of them were in the experimental group, 26 of them were in the control group. Among the 30 pre-service teachers, 25 (83.3%) of them were female and 5 (16.7%) of them were male. Among the 26 pre-service teachers, 18 (69.2%) of them were female and 8 (30.8%) of them were male. Random selection of participants from the population was not possible since different sections of the course were chosen by the pre-service teachers according to their own schedule at the beginning of the semester. However, the equality of experiment and control groups was checked by the application of “Principles and Methods of Instruction” course pre-achievement test at the beginning of the semester in order to strengthen the design, as stated by Cohen, Manion, and Morrison (2007). The results of the independent samples t test showed that the knowledge of the pre-service teachers about the “Principles and Methods of Instruction” course were not statistically significant in both the experimental (M= 32.33, SD = 6.60) and control groups (M = 35.67, SD = 8.70), t(56) = -1.63, p = .11. Thus, it can be said that both groups were equal at the beginning of the treatment in terms of achievement. Data collection instruments Data were collected through the implementation of an achievement test and a scale. The properties of data collection instruments are explained below. Achievement test (AT) A 40-question AT was implemented as pre- and post-test in both the experimental and control groups. The test included multiple-choice type questions except for the last question. The last question was a matching type question and included five items. The mean item difficulty of the test was 0.51, mean item discrimination index was 0.37 and the Kr-20 reliability coefficient was 0.78. The Classroom Environment Perceptions Scale of Pre-Service Teachers (CEPSPT) CEPSPT was implemented after the treatment as a post-test in both the experimental and control groups. It consisted of Likert-type items related to pre-service teachers’ perceptions of the classroom environment. We developed the instrument (Özüdoğru & Aksu, 2019) in accordance with constructivism (Taylor et al., 1997), environment fit theory (Fraser, 1998; Trickett & Moos, 1973), flipped learning principles (Bergmann & Sams, 2012) and also the works of Fraser and other researchers (Fraser, 1980; Fraser et al., 1992; Fraser & Treagust, 1986; Rentoul & Fraser, 1980; Taylor et al., 1997; Trickett & Moos, 1973). In the pilot study, the factor structure of the scale CEPSPT was determined through exploratory factor analysis; confirmatory factor analysis and reliability coefficients were calculated afterwards. The participants were selected from pre-service teachers studying at three universities in Turkey, which were not included in the actual study. A total of 528 pre-service teachers who were sophomores and took the “Principles and Methods of Instruction” course in the spring semester of 2016-2017 education year filled out the CEPSPT from May to June in 2016. Pre-service teachers needed 10–15 minutes to fill out the scale. After eliminating eight questionnaires due to unanswered items, 520 questionnaires were included for the analysis. Among the 520 pre-service teachers, 314 (60.4%) of them were female and 206 (39.6%) of them were male; 251 (48.3%) of them were from Uşak University, 239 (46%) of them from Ege University and 30 (5.8%) of them from Manisa Celal Bayar University in Turkey. Australasian Journal of Educational Technology, 2020, 36(4). 32 The final form of CEPSPT consisted of a 38-item, 5-point Likert scale (ranging from 1 – completely disagree to 5 – completely agree) regarding six scales: • The satisfaction scale consisted of 10 items (e.g., Students look forward to coming to this course). • The cooperation scale consisted of 7 items (e.g., Each student tries to fulfill his/her duties fully in individual or group work). • Th involvement scale consisted of 7 items (e.g., Students strive to complete the activities that are being conducted in class). • The task orientation scale consisted of 5 items (e.g., The learning tasks are planned clearly and carefully). • The student cohesiveness scale consisted of 5 items (e.g., Students know each other well). • The difficulty scale consisted of 4 items (e.g., Students are challenged in group work conducted in class.). Reliability was examined by Cronbach’s alpha coefficients of internal consistency which were calculated for each sub-scale of the CEPSPT. The reliability coefficient was found 0.85 for satisfaction, 0.84 for cooperation, 0.77 for involvement, .83 for task orientation, 0.77 for student cohesiveness, and 0.72 for difficulty scales, which are higher than 0.70 (Hair, Black, Babin, & Anderson, 2014). Moreover, the reliability coefficient of the scale as a whole was also calculated as 0.91. Student performance evaluation rubrics Three rubrics were developed by the researchers to score the performances of the pre-service teachers for different activities. The first rubric was about evaluating the weekly video homework. The second rubric was about evaluating a teaching technique presentation plan. Finally, the third rubric was about evaluating the teaching technique presentations of pre-service teachers. The last rubric consisted of two parts. The first part included criteria to evaluate the applications of pre-service teachers and was named “Application Skills” (50 points). The second part of the rubric included criteria to evaluate the communication and presentation skills of pre-service teachers and was named “Presentation Skills” (50 points). Data collection procedures and materials Before conducting the study, the approval of the Human Subjects Ethics Committee at Middle East Technical University was obtained. At the beginning of the semester, before conducting the study, the aim of the study and procedures that would be conducted during the semester were explained to the pre-service teachers and their permissions were sought. All pre-service teachers had the right not to participate in the study or withdraw from the study at any time they wanted. Their names were not revealed, and the data collected from them remained confidential. The procedures which were conducted in the experimental and control groups are explained in Table 1. The experimental group was taught by using flipped learning principles and provided with videos developed by the first author using Camtasia Studio 8 software before the course through Facebook and Edmodo. Each video lesson included pop-up questions to test whether they had learned the subject. The pop-up questions were inserted using H5P interactive videos. H5P includes pop quizzes and interactive timelines, which motivate pre-service teachers to be more active and engaged and increase the ratio of pre- service teachers watching videos. In the face-to-face part of the course, the pre-service teachers were not given additional lecturing or re-teaching of the video content but some important points that they did not understand were explained. As it can be seen in Table 1, there were Kahoot or Socrative applications, which are online question- answer game activities for the pre-service teachers who were in the experimental group. They were included as formative evaluation techniques to assess what the pre-service teachers had learned from the course videos. Moreover, the pre-service teachers took part in cooperative and collaborative learning activities in groups. They discussed the subject in groups, asked questions, explained their views, shared their ideas with group members, prepared posters, concept maps and sample lesson plans, conducted micro-teaching presentations and finally presented them to their classmates in groups. Also, they were provided with different teaching materials such as videos, textbooks and student response systems for Australasian Journal of Educational Technology, 2020, 36(4). 33 better learning opportunities. The materials and software which were included in the study are shown in Table 2. Finally, the pre-service teachers uploaded the classroom assignments to Moodle to be evaluated. Table 1 Procedures in the experimental and control groups Before class Inside the class Experimental group  Pre-service teachers were provided with videos before the course.  Pre-service teachers summarised the content of the videos.  Pre-service teachers reflected about the content.  Pre-service teachers came to class with at least one question about the subject.  Pre-service teachers interacted using Facebook, WhatsApp or Edmodo to help each other solve their problems.  Pre-service teachers uploaded the summary papers to Edmodo to be evaluated.  Pre-service teachers’ questions and the points they wondered about the content of videos were answered.  Experimental group was directed to Kahoot or Socrative applications in relation to the course content.  Group work was completed in the class requiring pre-service teachers to reflect on, discuss, and practise what they have learned. Control group  Pre-service teachers were assigned a topic and expected to read about it before the class.  Pre-service teachers uploaded the previous week’s homework to Edmodo to be evaluated.  Lecturing took place.  During lecturing, question and answering part took place.  Instructor answered pre-service teachers’ questions.  The Kahoot and pop-up questions were asked orally during the course. Table 2 Materials and software in the study and their functions Materials and software Functions of materials and software Videos  Present the course content  Test learning through interactive questions Camtasia 8.8  Prepare interactive videos H5P  Insert pop-up questions and video links from YouTube and the Internet Social networking sites – Facebook, WhatsApp  Ensure communication in the course for the experimental group  Announce the publication of videos for the experimental group  Share the names of top 3 pre-service teachers after the Kahoot or Socrative applications  Share the photos of pre-service teachers after group studies while preparing weekly assignments at the experimental group Kahoot, Socrative  Increase lesson engagement in the experimental group  Make enrichment activities and increase the retention level of pre- service teachers in the experimental group Moodle, Edmodo  Publish videos for the experimental group  Upload reflections and assignments for the experimental group and homework for the control group  Provide interaction among experimental group pre-service teachers to ask questions and examine individual weekly reflections The control group was taught according to the principles of traditional instruction. The pre-service teachers in the control group were assigned homework from their course books to read before the class time. The Australasian Journal of Educational Technology, 2020, 36(4). 34 course subject was presented during class time by the first author using PowerPoint slides. During presentations in the classroom, the control group were asked the pop-up questions included in the lecture videos and those from the Kahoot or Socrative activities. However, they usually worked on the assignments and tasks on their own outside of the class; the experimental group, however, completed them in class. Data analysis Data were analysed through descriptive and inferential statistical analysis techniques. The multivariate analysis of variance, MANOVA, was used to examine whether flipped learning affected the pre-service teachers’ AT scores, final grades and also their perceptions of the classroom environment (Hair et al., 2014; Tabachnick & Fidell, 2007). For the analysis of quantitative data, SPSS 22.0 was employed, and the alpha level was determined as .05. Before conducting MANOVA, the assumptions of univariate, multivariate normality, homogeneity of variance, homogeneity of covariance, and outliers were checked in order to explore the appropriateness of the data for MANOVA (Field, 2009; Tabachnick & Fidell, 2007). Results In this part, the quantitative results which were obtained from the AT and CEPSPT are presented. Effect of instruction on the AT and final grades MANOVA was conducted in order to examine whether the instruction type affected the pre-service teachers’ AT scores and final grades. The pre-service teachers’ final grades were calculated as 20% of the total scores for weekly video homework, 30% of the scores obtained from a teaching technique presentation and plan and 50% of the scores obtained from the AT. The pre-service teachers who obtained final grades of between 70 and 75 were deemed to have a low achievement score; those who obtained final grades of between 76 and 80 were deemed to have a medium achievement score and those who obtained final grades over 81 were deemed to have a high achievement score. Before interpreting the results of MANOVA, descriptive statistics are shown in Table 3. Table 3 Mean scores and standard deviations for the results of the AT and final grades Group AT Final grades M SD M SD Flipped learning 73.27 9.22 82.80 5.90 Traditional instruction 62.08 9.53 74.13 6.53 According to the results of the descriptive statistics as shown in Table 3, the pre-service teachers who were in the experimental group (M = 73.27, SD = 9.22) had higher achievement test scores than those in the control group (M = 62.08, SD = 9.53). Also, the pre-service teachers in the flipped learning group had higher final grades (M = 82.80, SD = 5.90) than those in the traditional instruction group (M = 74.13, SD = 6.53). According to the multivariate test results, Wilks’ Lambda test was significant, as shown in Table 4. The independent variable group had a significant effect on dependent variables F (2, 53) = 13.55 p < .05. After checking the multivariate results of MANOVA, univariate analyses were also interpreted. Before checking the ANOVA results, Bonferroni correction was conducted by dividing the alpha value of .05 by the number of dependent variables (.05/2 = .025). According to the test between subject results, it was seen that the group variable had a significant effect on both dependent variables. In other words, the pre-service teachers in the flipped learning group had significantly higher AT scores F (1, 54) = 27.21, p < .025 and final grades F (1, 54) = 19.89 p < .025 than the pre-service teachers in the traditional group. In other words, the pre- service teachers in the flipped learning group had significantly higher AT scores and final grades than those in the traditional instruction group. The multivariate eta squared .39 indicated that 39% of the multivariate variance of the AT scores and final grades were related to the group factor. Australasian Journal of Educational Technology, 2020, 36(4). 35 Table 4 Multivariate and univariate analyses of variance for the AT and final grades Variable MANOVA* F (2,53) ANOVA** F (1,54) AT Final grades Group 13.55 27.21 19.89 *p < .05 **p < .025 Effect of instruction on pre-service teachers’ perceptions of the classroom environment MANOVA was conducted to answer the second research question, which was proposed to investigate whether there was a significant difference between the pre-service teachers’ perceptions of the classroom environment scores (satisfaction, cooperation, involvement, task orientation, student cohesiveness, and difficulty) between those in the flipped learning and control groups. In this analysis, there were six dependent variables and one independent variable. Before interpreting the results of MANOVA, descriptive statistics are presented in Table 5. According to the results of the descriptive statistics shown in Table 5, the pre-service teachers who participated in flipped learning and traditional instruction groups perceived the classroom environment almost similarly. In other words, they had similar mean scores according to all dimensions of the classroom environment. The pre-service teachers in the flipped learning group had a mean M = 41.03 (SD = 4.99) for satisfaction; those in the traditional instruction group had a mean M = 42.33 (SD = 3.91). On the other hand, the pre-service teachers in the flipped learning group had a mean M = 21.47 (SD = 3.14) for student cohesiveness; those in the traditional instruction group had a mean M = 20.96 (SD = 2.57). Table 5 Mean scores and standard deviations for perceptions of the classroom environment G ro up Sa tis fa ct io n C oo pe ra tio n In vo lv em en t T as k or ie nt at io n St ud en t co he si ve ne ss D if fi cu lty M SD M SD M SD M SD M SD M SD Flipped learning 41.03 4.99 29.30 3.71 30.17 3.13 22.27 2.30 21.47 3.14 8.93 2.50 Traditional instruction 42.23 3.91 29.88 3.68 30.85 2.46 22.81 1.96 20.96 2.57 8.35 3.44 According to the multivariate test results as shown in Table 6, the independent variable group had no significant effect on the dependent variables F (6, 49) = .63 p > .05. After checking the multivariate results of MANOVA, univariate analyses were also interpreted. Before checking the ANOVA results, Bonferroni correction was conducted through dividing the alpha value of .05 by the number of dependent variables (.05/6 = .008). According to the test between subject results, it was seen that the group variable did not have any significant effect on the dependent variables. In other words, being in a flipped learning group or in a traditional group had no significant effect on the pre-service teachers’ perception of the classroom environment, namely satisfaction F (1, 54) = .98 p > .008; cooperation F (1, 54) = .35 p > .008; involvement F (1, 54) = .80 p > .008; task orientation F (1, 54) = .88 p > .008; student cohesiveness F (1, 54) = .43 p > .008 and difficulty F (1, 54) = .54 p > .008. Australasian Journal of Educational Technology, 2020, 36(4). 36 Table 6 Multivariate and univariate analyses of variance for perceptions of the classroom environment in flipped learning and traditional instruction groups Variable MANOVA* F (6,49) ANOVA** F (1,54) Sa tis fa ct io n C oo pe ra tio n In vo lv em en t T as k or ie nt at io n St ud en t co he si ve ne ss D if fi cu lty Group .63 .98 .35 .80 .88 .43 .54 *p > .05. **p >.008 Discussion and conclusion In this section, the findings of the current research are discussed in detail. Firstly, the findings related to the pre-service teachers’ achievement are discussed, then the findings obtained from the implementation of CEPSPT are discussed. The effect of flipped learning on pre-service teachers’ principles and methods of instruction course achievement According to the results of the study, the pre-service teachers in the flipped learning group obtained significantly higher AT scores and final grades than those in the traditional instruction group. This might have stemmed from the fact that flipped learning included educational technology and different learning activities, as shown in Figure 1; in this way it affected learning positively. The implementation of flipped learning might have provided more time in terms of pre-class preparation and active in-class participation for learning when compared to those in the traditional classroom. In this way, the pre-service teachers cooperated with others when completing class assignments and could learn from each other – which is in line with the literature (Butt, 2014; Caudill, 2014; Durik & Eccles, 2006; Findlay-Thompson & Mombourquette, 2014; Mason et al., 2013; Patrick, Ryan, & Kaplan, 2007; Walker, 2003; Wang, 2012). In addition to these, in this study, the inclusion of student response systems like Kahoot and Socrative might have motivated students and increased their performance, as also stated by Guess (2008). Cohn and Fraser (2015) stated that the inclusion of computer applications during instruction increased students’ understanding and retention; also, as stated by Ziegelmeier and Topaz (2015), they are a way to check students’ knowledge to keep them up to date with course material and encourage them to watch the videos before the face-to-face segment of the course. In this way, the involvement of students who used student response systems increased and they could self-evaluate their level of understanding of the material (Martyn, 2007). Moreover, the pre-service teachers in the flipped group watched lecture videos and while doing so, they had the opportunity of rewatching some parts whenever they needed, taking notes, writing questions to be asked in the class, which might have increased their achievement when compared to those in the control group; this is in line with previous research (Bergmann & Sams, 2012; Fulton, 2012). Similarly to this study, the results of the study conducted by Yoshida (2016) with 66 Japanese pre-service teachers who took an educational technology course to find out what they found useful about flipped learning revealed that studying through videos over and over again, studying at their own pace and whenever they wanted and stopping the video whenever it was necessary were found useful because they enhanced learning and contributed to the effectiveness of the face-to-face part of the course. Furthermore, in this study, the instructor was able to monitor the flipped group’s performance and comprehension during in-class activities and immediately clarify any misunderstandings or confusion – which might be one of the reasons for the achievement of the flipped learning group in line with previous research (Bergmann & Sams, 2012; Berrett, 2012; Findlay-Thompson & Mombourquette, 2014; Ziegelmeier & Topaz, 2015). In addition to these, the pre-service teachers interacted with the instructor out of class by using social media or Edmodo, and the instructor could help them anytime they needed. Australasian Journal of Educational Technology, 2020, 36(4). 37 Furthermore, similar to this study, den Brok, Brekelmans, and Wubbels, (2004) indicated that interpersonal teacher behaviour is significantly related to the performance of students. Hence, it can be said that the increased faculty-student interaction might be one of the reasons for the achievement of the flipped learning group. All in all, it can be inferred from this study that changing the order of instruction is an improvement for the learning of pre-service teachers, as also stated by Talbert (2012), since while in flipped classes, instructors are available in class to help students achieve high-order learning such as analysis, synthesis and evaluation; however, this differs from traditional teacher training classes where time is taken up with tasks at the lowest cognitive levels (knowledge and understanding) levels of Bloom’s taxonomy (Bergmann & Sams, 2012; Talbert, 2012). The effect of flipped learning on pre-service teachers’ perceptions of the classroom environment According to the results obtained from the CEPSPT, the flipped learning and the traditional instruction groups did not have significantly different mean scores for all the dimensions of the classroom environment. In the study conducted by Roth (1998), it was found that negative classroom environment perceptions were related to lower achievement and positive classroom environment perceptions were related to higher achievement. In this study, although the pre-service teachers in the flipped learning group had higher final grades and AT scores than those who in the control group, both groups had similar classroom environment perceptions. The reason for this result might be the instructor support. Both groups were taught by the same instructor, and they knew that whenever they had problems, they would be welcomed and the problems solved by the instructor. Moreover, when the pre-service teachers in both groups discussed the topics in class, they were asked questions and explained their opinions about the questions. In other words, their ideas and suggestions were welcomed without any judgment and used during class discussions. Hence, it can be said that the pre-service teachers of both groups perceived that they were cared for by the instructor and felt the satisfaction and involvement properties of the classroom environment at similar levels. In this study, the pre-service teachers in both the flipped learning and control groups perceived the satisfaction property of the classroom environment similarly. Many studies have reported that flipped learning increases students’ satisfaction (Lage et al., 2000; Lee, Lim, & Kim, 2016; Touchton, 2015). On the other hand, some researchers have not been positive about whether the flipped learning environment increases the satisfaction of students (Butzler, 2014; Findlay-Thompson & Mombourquette, 2014; Missildine, Fountain, Summers, & Gosselin, 2013; Strayer, 2012; Zappe et al., 2009). In this study, technological issues such as slow Internet and the music behind the voice of the instructor in the course videos might have affected the satisfaction of the pre-service teachers. Also, the duration of the course videos might have been too long and not very interesting for some of the pre-service teachers, which might have affected the total satisfaction and involvement scores. Hence, the pre-service teachers in both the flipped learning and control groups had similar satisfaction and involvement scores related to the course. The pre-service teachers in both the flipped learning and control groups had similar classroom environment perceptions in terms of cooperation. The reason for this result might be that some of the group members in the flipped learning group might have remained passive during group work. When assignments were being completed in groups, if some of the group members remained passive, others might have needed to work more – as also revealed by the business management students in the study conducted by Chien (2007) – which may have decreased the motivation, cooperation, student cohesiveness and satisfaction properties of classroom environment for some of the pre-service teachers. The pre-service teachers in both the flipped learning and control groups perceived the task orientation property of the classroom environment at similar levels. On the other hand, classrooms characterised by high degrees of teacher expectation, direction, structure, and organisation tend to produce the most favourable cognitive and affective student outcomes (Chien, 2007; Chionh & Fraser, 2009; Dorman et al., 2003; Velayutham & Aldridge, 2013; Zandvliet, 1999). However, the self-directed learning properties of flipped learning, which require planned and systematic work, might not be appropriate for some pre-service teachers. They might have needed more support and guidance in using Moodle-Edmodo or taking part in computer-based activities, which might have affected their satisfaction and task orientation scores. As also stated by Mason et al. (2013), the pre-service teachers might have felt overwhelmed with the number of tasks conducted in the face-to-face part of the course and out of the course. Also, as stated by Strayer (2007), the statistics flipped learning group needed more reassurance that they were on the right track, as Australasian Journal of Educational Technology, 2020, 36(4). 38 they were left to use technology to learn and solve problems outside the classroom environment, in contrast to the students in a traditional lecture course. The pre-service teachers in both the flipped learning and control groups perceived the classroom environment similarly in terms of student cohesiveness. However, different from this study, research has indicated that the student cohesiveness dimension of the classroom environment is an important factor in explaining outcomes in various subject matter areas (Chionh & Fraser, 2009; Dorman et al., 2003; Fraser, 1998; Haertal et al., 1981; Majeed et al., 2002; Velayutham & Aldridge, 2013). In this study, the sophomore pre-service teachers in both groups might have already known each other by their first names, they might have been close enough to be personal friends, willing to get to know each other and help each other with the learning process. Hence, they had similar student cohesiveness perception of the classroom environment according to CEPSPT. The pre-service teachers in the flipped learning group perceived the classroom as difficult as those who were in the control group. Although flipped learning required both pre-class preparation and in-class active involvement, the pre-service teachers in the flipped learning group did not perceive these tasks as difficult and were as satisfied with the course as the pre-service teachers in the control group. Moreover, the inclusion of educational technology, as shown in Figure 1, might have decreased the perceived difficulty of the classroom environment by providing moderately challenging tasks, exciting learning experiences and a controlled race in activities and the feeling of fun while learning, which is in line with previous literature (Touchton, 2015; Zandvliet, 1999). Finally, one of the reasons for the similar classroom environment perceptions of the flipped learning and traditional instruction groups might have stemmed from the items of CEPSPT. The scale included 5-point Likert type (5 – strongly agree, 1 – strongly disagree) responses, which might not have provided a wide range of response options to reflect the difference between the perceptions of the pre-service teachers in both groups. Most pre-service teachers responded among a small range, which might have narrowed their perceptions about the classroom environment. Implications for practice Many studies have indicated insignificant differences in student learning when flipped learning is implemented (Butzler, 2014; Clark, 2013; Davies et al, 2013; Findlay-Thompson & Mombourquette, 2014; McLaughlin et.al, 2013; Ramaglia, 2015; Ziegelmeier & Topaz, 2015). The reason for this may be the different design processes implemented by different flipped learning researchers. If researchers do not give importance to pre-class preparation or in-class active learning tasks, the results may not be as expected, and students are likely to experience the classroom environment negatively and perceive less involvement, satisfaction, cooperation, cohesiveness, and teacher and peer support and experience more difficulty, which in turn affects student achievement. Hence, the principles of flipped learning that were used in this study can be recommended to practitioners for effectively designing classroom environments. Also, it is recommended that interaction between students and instructors should be provided through the use of social media in order to increase the involvement of pre-service teachers in the course, support pre-class studies and relieve the burden of learning. In addition, it is suggested that different computer applications, software and games should be included during the teaching-learning process in the flipped learning group in order to increase students’ satisfaction with classroom activities, which contribute to learning and more positive perceptions of the classroom environment. Furthermore, instructors should monitor pre-service teachers to ensure that they do not rely too much on others during group studies, since some might remain passive during group work. Also, instructors should be careful to monitor group studies during in-class tasks, otherwise students may go off-task and engage in social conversations rather than focusing only on the activity at hand. Furthermore, to be able to conduct different activities, it should be taken into consideration that pre-service teachers come to class prepared. If they are provided with longer videos, they may come to class without watching them, as revealed in the study conducted by Gaughan (2014). Hence, it is recommended that lecture videos should be 1½ minutes per grade level, as stated by Raths (2014), and should be “chunked” as they were in this study by including only the most important issues, rather than engaging in long lectures. In this way, pre-service teachers would find the content to be shorter and easier to understand. Australasian Journal of Educational Technology, 2020, 36(4). 39 Limitations and future work In summary, the findings of this study might be useful for instructors for design instruction by sensitising themselves to the positive and negative influences of the dimensions of the classroom environment in the flipped learning context. For clearer and deeper conclusions and educational implications, however, future research is needed. Based on the findings of this study, recommendations for future research are as follows. The results obtained in the current study were limited to the “Principles and Methods of Instruction” course, which is an obligatory course for all the departments in the faculties of education in Turkey. Hence, pre- service teachers might be motivated to learn and to complete both the out-of-class work and in-class activities in order not to fail in the course. For this reason, to examine the effectiveness of flipped learning, it is suggested that a different study might be set in an elective course. Moreover, some of the results obtained in this study are limited to the implementation of a scale, CEPSPT, including complex constructs such as satisfaction, involvement, task orientation and student cohesiveness, which might not be simply reduced to measurable items. Hence, the findings might be investigated through further research by involving qualitative methods such as observations in both flipped learning and traditional instruction environments. Furthermore, the results obtained are limited to the pre-service teachers’ perceptions of the actual classroom environment. Further research may investigate classroom environments according to pre- service teachers’ actual and preferred perceptions of classroom environments to identify discrepancies between the two learning environments and any necessary precautions that could be implemented to reduce discrepancies. Lastly, this study was limited to the data obtained from 56 pre-service teachers in total. Hence, it is suggested that the findings might be confirmed with a large sample size in other universities for generalisation. References Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1–14. https://doi.org/10.1080/07294360.2014.934336 Arısoy, N. (2007). Examining 8th grade students’ perception of learning environment of science classrooms in relation to motivational beliefs and attitudes (Master’s thesis). Middle East Technical University, Ankara, Turkey. Atbas, E. E. (2004). The effect of students’ entering characteristics and classroom environment experiences on their language learning outcomes in an EFL setting in Turkey (Doctoral dissertation). Middle East Technical University, Ankara, Turkey. Baker, J. W. (2000). The classroom flip: Using web course management tools to become the guide by the side. In J. A. Chambers (Ed.), Selected papers from the 11th International Conference on College Teaching and Learning (pp. 9–17). Jacksonville: Florida Community College at Jacksonville. Retrieved from http://www.classroomflip.com/files/classroom_flip_baker_2000.pdf Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. Alexandria, VA: International Society for Technology in Education. Berrett, D. (2012, February 19). How ‘flipping’ the classroom can improve the traditional lecture. The Chronicle of Higher Education. Retrieved from http://chronicle.com/article/How Flipping-the- Classroom/130857/ Bishop, J. L., & Verleger, M. A. (2013). The flipped classroom: A survey of the research. In Proceedings of the 120th ASEE Annual Conference & Exposition. Washington, DC: American Society for Engineering Education (ASEE) National Conference. Retrieved from http://www.asee.org/public/conferences/20/papers/6219/view Butt, A. (2014). Student views on the use of a flipped classroom approach: Evidence from Australia. Business Education & Accreditation, 6(1), 33–43. Retrieved from http://www.theibfr2.com/RePEc/ibf/beaccr/bea-v6n1-2014/BEA-V6N1-2014-4.pdf Butzler, K. B. (2014). The effects of motivation on achievement and satisfaction in a flipped classroom learning environment (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 3637765) Caudill, N. V. (2014). Pre-service teachers’ perceptions of a flipped classroom: A study of undergraduates enrolled in an applied child development course (Master’s thesis). Retrieved from https://repository.lib.ncsu.edu/bitstream/handle/1840.16/9699/etd.pdf https://doi.org/10.1080/07294360.2014.934336 http://www.classroomflip.com/files/classroom_flip_baker_2000.pdf http://chronicle.com/article/How%20Flipping-the-Classroom/130857/ http://chronicle.com/article/How%20Flipping-the-Classroom/130857/ http://www.asee.org/public/conferences/20/papers/6219/view http://www.theibfr2.com/RePEc/ibf/beaccr/bea-v6n1-2014/BEA-V6N1-2014-4.pdf https://repository.lib.ncsu.edu/bitstream/handle/1840.16/9699/etd.pdf Australasian Journal of Educational Technology, 2020, 36(4). 40 Chien, C. F. (2007). Development, validation and use of an instrument for assessing business management learning environments in higher education in Australia: The business management education learning environment inventory (BMELEI) (Doctoral dissertation). Retrieved from https://espace.curtin.edu.au/bitstream/handle/20.500.11937/1095/17957_Chien, Chee.pdf Chionh, Y. H., & Fraser, B. J. (2009). Classroom environment, achievement, attitudes and self-esteem in geography and mathematics in Singapore. International Research in Geographical and Environmental Education, 18(1), 29–44. https://doi.org/10.1080/10382040802591530 Clark, K. R. (2013). Examining the effects of the flipped model of instruction on student engagement and performance in the secondary mathematics classroom: An action research study (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 3592584) Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th ed.). New York, NY: Routledge. Cohn, S., & Fraser, B. (2015). Effectiveness of student response systems in terms of learning environment, attitudes and achievement. Learning Environments Research, 19(2), 163–172. https://doi.org/10.1007/s10984-015-9195-0 Davies, R. S., Dean, D. L., & Ball, N. (2013). Flipping the classroom and instructional technology integration in a college-level information systems spreadsheet course. Educational Technology Research and Development, 61(4), 563–580. https://doi.org/10.1007/s11423-013-9305-6 Day, J. A. & Foley, J. D. (2006). Evaluating a web lecture intervention in a human computer interaction course. IEEE Transactions on Education, 49(4), 420–431. https://doi.org/10.1109/TE.2006.879792 den Brok, P., Brekelmans, M., & Wubbels, T. (2004). Interpersonal teacher behavior and student outcomes. School Effectiveness and School Improvement, 15(3-4), 407–442. https://doi.org/10.1080/09243450512331383262 Dorman, J. (2002). Classroom environment research: Progress and possibilities. Queensland Journal of Educational Research, 18(2), 112–140. Retrieved from http://iier.org.au/qjer/qjer18/dorman.html Dorman, J. P., Adams, J. D., & Ferguson, J. M. (2003). A cross-national investigation of students’ perceptions of mathematics classroom environment and academic efficacy in secondary schools. International Journal for Mathematics Teaching and Learning. Retrieved from http://www.cimt.org.uk/journal/dormanj.pdf Dotterer, A. M. & Lowe, K. (2011). Classroom context, school engagement, and academic achievement in early adolescence. Journal of Youth and Adolescence, 40(12), 1649–1660. https://doi.org/10.1007/s10964-011-9647-5 Durik, A. M., & Eccles, J. S. (2006). Classroom activities in math and reading in early, middle, and late elementary school. Journal of Classroom Interaction, 41(1), 33–41. Retrieved from https://www.jstor.org/stable/23869757 Field, A. P. (2009). Discovering statistics using SPSS (3th ed.). London, United Kingdom: Sage. Findlay-Thompson, S., & Mombourquette, P. (2014). Evaluation of a flipped classroom in an undergraduate business course. Business Education & Accreditation, 6(1), 63–71. Retrieved from https://www.theibfr.com/download/BEA/2014-bea/bea-v6n1-2014/BEA-V6N1-2014.pdf Foertsch, J., Moses, G., Strikwerda, J., & Litzkow, M. (2002). Reversing the lecture/homework paradigm using eTEACH web-based streaming video software. Journal of Engineering Education, 91(3), 267– 274. https://doi.org/10.1002/j.2168 9830.2002.tb00703.x Fraser, B. J. (1980). Criterion validity of an individualized classroom environment questionnaire. Retrieved from ERIC database. (ED214961) Fraser, B. J. (1986). Classroom environment. London, United Kingdom: Croom Helm. Fraser, B. J. (1998). Classroom environment instruments: Development, validity, and applications. Learning Environments Research, 1(1), 7–33. https://doi.org/10.1023/A:1009932514731 Fraser, B. J., Giddings, G. J., & McRobbie, C. J. (1992). Assessment of the psychosocial environment of university science laboratory classrooms: A cross-national study. Higher Education, 24, 431–451. https://doi.org/10.1007/BF00137241 Fraser, B. J., & Treagust, D. F. (1986). Validity and use of an instrument for assessing classroom psychosocial environment in higher education. Higher Education, 15, 37–57. Retrieved from http://www.jstor.org/stable/3446741 Fulton, K. P. (2012). Upside down and inside out: Flip your classroom to improve student learning. Learning & Leading with Technology, 39(8), 12–17. Retrieved from ERIC database. (EJ982840) Gaughan, J. E. (2014). The flipped classroom in world history. History Teacher, 47(2), 221–244. Retrieved from http://www.societyforhistoryeducation.org/pdfs/F14_Gaughan.pdf https://espace.curtin.edu.au/bitstream/handle/20.500.11937/1095/17957_Chien,%20Chee.pdf https://doi.org/10.1080/10382040802591530 https://doi.org/10.1007/s10984-015-9195-0 https://doi.org/10.1109/TE.2006.879792 https://doi.org/10.1080/09243450512331383262 http://iier.org.au/qjer/qjer18/dorman.html http://www.cimt.org.uk/journal/dormanj.pdf https://doi.org/10.1007/s10964-011-9647-5 https://www.jstor.org/stable/23869757 https://www.theibfr.com/download/BEA/2014-bea/bea-v6n1-2014/BEA-V6N1-2014.pdf https://doi.org/10.1002/j.2168%209830.2002.tb00703.x https://doi.org/10.1023/A:1009932514731 https://doi.org/10.1007/BF00137241 http://www.jstor.org/stable/3446741 http://www.societyforhistoryeducation.org/pdfs/F14_Gaughan.pdf Australasian Journal of Educational Technology, 2020, 36(4). 41 Goh, S. C., & Fraser, B. J. (2000). Teacher interpersonal behavior and elementary students’ outcomes. Journal of Research in Childhood Education, 14(2), 216–231. https://doi.org/10.1080/02568540009594765 Guess, A. (2008). Keeping clickers in the classroom. Inside Higher Education. Retrieved from http://www.insidehighered.com/news/2008/07/18/keeping-clickers-classroom Haertel, G.D., Walberg, H.J., & Haertel, E. H. (1981). Socio-psychological environments and learning: a quantitative synthesis. British Educational Research Journal, 7(1), 27–36. https://doi.org/10.1080/0141192810070103 Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis (7th ed.). London, United Kingdom: Pearson. Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2014). NCM Horizon Report: 2014 Higher education edition. Austin, TX: The New Media Consortium. Retrieved from https://library.educause.edu/-/media/files/library/2014/1/hr2014-pdf.pdf Kim, H. B., Fisher, D. L., & Fraser, B. J (2000). Classroom environment and teacher interpersonal behavior in secondary science classes in Korea. Evaluation & Research in Education, 14(1), 3–22. https://doi.org/10.1080/09500790008666958 Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. Journal of Economic Education, 31(1), 30–43. https://doi.org/10.1080/00220480009596759 LaRocque, M. (2008). Assessing perceptions of the environment in elementary classrooms: The link with achievement. Educational Psychology in Practice, 24(4), 289–305. https://doi.org/10.1080/02667360802488732 Lee, C. K. J., Lee, L. M. F., & Wong, H. W. (2003). Development of a classroom environment scale in Hong Kong. Educational Research and Evaluation, 9(4), 317–344. https://doi.org/10.1076/edre.9.4.317.17813 Lee, J., Lim, C., & Kim, H. (2016). Development of an instructional design model for flipped learning in higher education. Education Technology Research and Development, 65(2), 427– 453. https://doi.org/10.1007/s11423-016-9502-1 Love, B., Hodge, A., Grandgenett, N., & Swift, A. W. (2014). Student learning and perceptions in a flipped linear algebra course. International Journal of Mathematical Education in Science and Technology, 45(3), 317–324. https://doi.org/10.1080/0020739X.2013.822582 Majeed, A., Fraser, B. J., & Aldridge, J. M. (2002). Learning environment and its association with student satisfaction among mathematics students in Brunei Darussalam. Learning Environments Research, 5(2), 203–226. https://doi.org/10.1023/A:1020382914724 Manisa Celal Bayar University. (2019). About MCBU. Retrieved from http://en.cbu.edu.tr/general- overview/about-cbu.5009.en.html Martyn, M. (2007). Clickers in the classroom: An active learning approach. Educase Quarterly, 30(2), 71–74. Retrieved from https://er.educause.edu/~/media/files/article%20downloads/eqm0729.pdf Mason, G. S., Shuman, T. R., & Cook, K. E. (2013). Comparing the effectiveness of an inverted classroom to a traditional classroom in an upper-division engineering course. IEEE Transactions on Education, 56(4), 430–435. https://doi.org/10.1109/TE.2013.2249066 McLaughlin, J. E., Griffin, L. M., Esserman, D. A., Davidson, C. A., Glatt, D. M., Roth, M. T., … Mumper, R. J. (2013). Pharmacy student engagement, performance, and perception in a flipped satellite classroom. American Journal of Pharmaceutical Education, 77(9), 1–8. https://doi.org/10.5688/ajpe779196 McRobbie, C. J., Roth, W. M., & Lucus, K. B. (1997). Multiple learning environments in the physics classroom. International Journal of Educational Research, 27(4), 333–342. https://doi.org/10.1016/S08830355(97)90015-X Millard, E. (2012). 5 reasons flipped classrooms work: Turning lectures into homework to boost student engagement and increase technology fueled creativity. University Business, 15(11), 26–29. Retrieved from https://www.questia.com/library/p1205/university-business/i3164168/vol-15-no-11-december Missildine, K., Fountain, R., Summers, L., & Gosselin, K. (2013). Flipping the classroom to improve student performance and satisfaction. Journal of Nursing Education, 52(10), 597–599. https://doi.org/10.3928/01484834-20130919-03 Murphree, D. S. (2014). Writing wasn’t really stressed, accurate historical analysis was stressed: Student perceptions of in-class writing in the inverted, general education, university history survey course. The History Teacher, 47(2), 209–219. Retrieved from http://www.societyforhistoryeducation.org/pdfs/F14_Murphree.pdf https://doi.org/10.1080/02568540009594765 http://www.insidehighered.com/news/2008/07/18/keeping-clickers-classroom https://doi.org/10.1080/0141192810070103 https://library.educause.edu/-/media/files/library/2014/1/hr2014-pdf.pdf https://doi.org/10.1080/09500790008666958 https://doi.org/10.1080/00220480009596759 https://doi.org/10.1080/02667360802488732 https://doi.org/10.1076/edre.9.4.317.17813 https://doi.org/10.1007/s11423-016-9502-1 https://doi.org/10.1080/0020739X.2013.822582 https://doi.org/10.1023/A:1020382914724 http://en.cbu.edu.tr/general-overview/about-cbu.5009.en.html http://en.cbu.edu.tr/general-overview/about-cbu.5009.en.html https://er.educause.edu/%7E/media/files/article%20downloads/eqm0729.pdf https://doi.org/10.1109/TE.2013.2249066 https://doi.org/10.5688/ajpe779196 https://doi.org/10.1016/S08830355(97)90015-X https://www.questia.com/library/p1205/university-business/i3164168/vol-15-no-11-december https://doi.org/10.3928/01484834-20130919-03 http://www.societyforhistoryeducation.org/pdfs/F14_Murphree.pdf Australasian Journal of Educational Technology, 2020, 36(4). 42 O’Reilly, R. (1975). Classroom climate and achievement in secondary school mathematics classes. Alberta Journal of Educational Research, 21(4), 241–248. Retrieved from ERIC database. (ED101473) Özüdoğru, M., & Aksu, M. (2019). The development of classroom environment perceptions scale for pre- service teachers. Ankara University Journal of Faculty of Educational Sciences, 52(3), 773–794. https://doi.org/10.30964/auebfd.487734 Patrick, H., Ryan, A. M., & Kaplan, A. (2007). Early adolescents’ perceptions of the classroom social environment, motivational beliefs, and engagement. Journal of Educational Psychology, 99(1), 83– 98. https://doi.org/10.1037/0022-0663.99.1.83 Phillips, C. R., & Trainor, J. E. (2014). Millennial students and the flipped classroom. Proceedings of the ASBBS Annual Conference, 21(1), 519–530. Retrieved from http://asbbs.org/files/ASBBS2014/PDF/P/Phillips_Trainor(P519-530).pdf Ramaglia, H. (2015). The flipped mathematics classroom: A mixed methods study examining achievement, active learning, and perception (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (UMI No. 10002822) Raths, D. (2014). Nine video tips for a better flipped classroom. Education Digest, 79(6), 15–22. Retrieved from https://thejournal.com/articles/2013/11/18/9-video-tips-for-a-better-flipped- classroom.aspx Rentoul, A. J., & Fraser, B. J. (1980). Predicting learning from classroom individualization and actual preferred congruence. Studies in Educational Evaluation, 6(3), 265–277. https://doi.org/10.1016/0191 491X(80)90029-2 Roth, W. M. (1998). Teacher-as-researcher reform: Student achievement and perceptions of learning environment. Learning Environments Research, 1(1), 75–93. https://doi.org/10.1023/A:1009988732478 Schwab, J. J. (1969). The practical: A language for curriculum. The School Review 78(1), 1–23. Strayer, J. (2007). The effects of the classroom flip on the learning environment: A comparison of learning activity in a traditional classroom and a flip classroom that used an intelligent tutoring system (Doctoral dissertation). The Ohio State University, Columbus. Strayer, J. (2012). How learning in an inverted classroom influences cooperation, innovation and task orientation. Learning Environments Research, 15(2), 171–193. https://doi.org/10.1007/s10984-012 9108-4 Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (6th ed.). Boston, MA: Pearson. Talbert, R. (2012). Inverted classroom. Colleagues, 9(1), 1–3. Retrieved from http://scholarworks.gvsu.edu/colleagues/vol9/iss1/7 Talbert, R. (2014). Inverting the linear algebra classroom. Primus, 24(5), 361–374. https://doi.org/10.1080/10511970.2014.883457 Talley, C., & Scherer, S. (2013). The enhanced flipped classroom: increasing academic performance with student-recorded lectures and practice testing in a flipped stem course. Journal of Negro Education, 82(3), 339–347. https://doi.org/10.7709/jnegroeducation.82.3.0339 Taylor, P. C., Fraser, B. J., & Fisher, D. L. (1997). Monitoring constructivist classroom learning environments. International Journal of Educational Research, 27(4), 293–302. https://doi.org/10.1016/S0883-0355(97)90011-2 Touchton, M. (2015). Flipping the classroom and student performance in advanced statistics: Evidence from a quasi-experiment. Journal of Political Science Education, 11(1), 28–44. https://doi.org/10.1080/15512169.2014.985105 Trickett, E. J., & Moos, R. H. (1973). Social environment of junior high and high school classrooms. Journal of Educational Psychology, 65(1), 93-102. https://doi.org/10.1037/h0034823 Tune, J. D., Sturek, M., & Basile, D. P. (2013). Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Advances in Physiology Education, 37(4), 316–320. https://doi.org/10.1152/advan.00091.2013 Velayutham, S., & Aldridge, J. M. (2013). Influence of psychosocial classroom environment on students’ motivation and self-regulation in science learning: A structural equation modelling approach. Research in Science Education, 43, 507–527. https://doi.org/10.1007/s11165-011-9273-y Walker, S. L. (2003). Development and validation of an instrument for assessing distance education learning environments in higher education: The distance education learning environments survey (deles) (Doctoral dissertation). Curtin University of Technology, Perth, Australia. https://doi.org/10.30964/auebfd.487734 https://doi.org/10.1037/0022-0663.99.1.83 http://asbbs.org/files/ASBBS2014/PDF/P/Phillips_Trainor(P519-530).pdf https://thejournal.com/articles/2013/11/18/9-video-tips-for-a-better-flipped-classroom.aspx https://thejournal.com/articles/2013/11/18/9-video-tips-for-a-better-flipped-classroom.aspx https://doi.org/10.1016/0191%20491X(80)90029-2 https://doi.org/10.1016/0191%20491X(80)90029-2 https://doi.org/10.1023/A:1009988732478 https://doi.org/10.1007/s10984-012%209108-4 https://doi.org/10.1007/s10984-012%209108-4 http://scholarworks.gvsu.edu/colleagues/vol9/iss1/7 https://doi.org/10.1080/10511970.2014.883457 https://doi.org/10.7709/jnegroeducation.82.3.0339 https://doi.org/10.1016/S0883-0355(97)90011-2 https://doi.org/10.1080/15512169.2014.985105 https://doi.org/10.1037/h0034823 https://doi.org/10.1152/advan.00091.2013 https://doi.org/10.1007/s11165-011-9273-y Australasian Journal of Educational Technology, 2020, 36(4). 43 Wang, M. T. (2012). Educational and career interests in math: A longitudinal examination of the links between perceived classroom environment, motivational beliefs, and interests. Developmental Psychology, 48(6), 1643–1657. https://doi.org/10.1037/a0027247 Wilson, S. G. (2013). The flipped class: A method to address the challenges of an undergraduate statistics course. Teaching of Psychology, 40(3), 193–199. https://doi.org/10.1177/0098628313487461 Yoshida, H. (2016). Perceived usefulness of flipped learning on instructional design for elementary and secondary education: With focus on pre-service teacher education. International Journal of Information and Education Technology, 6(6), 430–434. https://doi.org/10.7763/IJIET.2016.V6.727 Yüksek Öğretim Kurulu Yayını. (2007). Öğretmen yetiştirme ve eğitim fakülteleri (1982-2007): Öğretmenin üniversitede yetiştirilmesinin değerlendirilmesi [Teacher training and faculties of education (1982-2007): The evaluation of teacher training in university]. Ankara, Turkey: Author. Zandvliet, D. B. (1999). The physical and psychosocial environment associated with classrooms using new information technologies: A cross-national study (Doctoral dissertation). Retrieved from https://espace.curtin.edu.au/bitstream/handle/20.500.11937/2124/9959_Zandvliet, D. 1999.pdf Zappe, S., Leicht, R., Messner, J., Litzinger, T., & Lee, H. (2009). Flipping the classroom to explore active learning in a large undergraduate course. ASEE Annual Conference and Exposition, Conference Proceedings. Retrieved from https://peer.asee.org/flipping-the-classroom-to-explore-active-learning- in-a-large-undergraduate-course.pdf Ziegelmeier, L. B., & Topaz, C. M. (2015). Flipped calculus: A study of student performance and perceptions. Primus, 25(9-10), 847–860. https://doi.org/10.1080/10511970.2015.1031305 Corresponding author: Melike Özüdoğru, ozudogru123@gmail.com Copyright: Articles published in the Australasian Journal of Educational Technology (AJET) are available under Creative Commons Attribution Non-Commercial No Derivatives Licence (CC BY-NC-ND 4.0). Authors retain copyright in their work and grant AJET right of first publication under CC BY-NC-ND 4.0. Please cite as: Özüdoğru, M., & Aksu, M. (2020). Pre-service teachers’ achievement and perceptions of the classroom environment in flipped learning and traditional instruction classes. Australasian Journal of Educational Technology, 36(4), 27–43. https://doi.org/10.14742/ajet.5115 https://doi.org/10.1037/a0027247 https://doi.org/10.1177/0098628313487461 https://doi.org/10.7763/IJIET.2016.V6.727 https://espace.curtin.edu.au/bitstream/handle/20.500.11937/2124/9959_Zandvliet,%20D.%20%201999.pdf https://doi.org/10.1080/10511970.2015.1031305 mailto:ozudogru123@gmail.com https://creativecommons.org/licenses/by-nc-nd/4.0/ https://doi.org/10.14742/ajet.5115 Method Research design Educational context Course information Participants of the study Data collection instruments Achievement test (AT) The Classroom Environment Perceptions Scale of Pre-Service Teachers (CEPSPT) Student performance evaluation rubrics Data collection procedures and materials Data analysis Results Effect of instruction on the AT and final grades Effect of instruction on pre-service teachers’ perceptions of the classroom environment Discussion and conclusion The effect of flipped learning on pre-service teachers’ principles and methods of instruction course achievement The effect of flipped learning on pre-service teachers’ perceptions of the classroom environment Implications for practice References