a © 2022 Indonesian Society for Science Educator 469 J.Sci.Learn.2022.5(3).469-477 Received: 13 January 2022 Revised: 23 April 2022 Published: 27 November 2022 Online Flipped Classroom: Developing Postgraduate Science Education Students’ Critical Thinking Skills Wahono Widodo1* 1Department of Science Education, Faculty of Mathematics And Natural Sciences, Universitas Negeri Surabaya, Surabaya, Indonesia *Corresponding author: wahonowidodo@unesa.ac.id ABSTRACT During the Covid-19 pandemic, online learning should be carried out through innovative methods. This study aims to determine whether implementing an online flipped classroom, with converting all face-to-face sessions into virtual face-to-face sessions, can facilitate the development of critical thinking skills of postgraduate science education students in the School Physics course. This quantitative study used one group pretest-postest research design. The subjects were postgraduate science education students who took School Physics courses in the first semester at one of the public universities in Indonesia. Online Flipped classroom was supported by Google Classroom Learning Management System (LMS). Critical thinking skills data in School Physics were obtained through tests. Data were analyzed using a description of normalized changes (c) and nonparametric-inferential analysis with the Related-Samples Wilcoxon Signed Rank Test. The results showed that the online flipped classroom could facilitate the development of the students’ critical thinking skills with an average normalized change of .71 in a high category. Even though the students felt the benefits of an online flipped classroom, they still wanted a face-to-face learning mode to be carried out soon. Keywords Online flipped classroom, Critical thinking skills, School Physics, Postgraduate science education students. 1. INTRODUCTION Higher education is responsible for developing students’ critical thinking skills (Bassham, Irwin, Nardone, & Wallace, 2013). Necessary thinking skills are essential in coping with academic or professional interests (Costa & Kallick, 2014; Kraisuth & Panjakajornsak, 2017) and become an indicator of lifelong learning (OECD, 2018). Postgraduate students majoring in Science Education are prepared to become academics who can later act as researchers, lecturers, educational consultants, or teacher advisors. As shown in the overview of The University of British Columbia's Master of Education in Science Education (MEd), the postgraduate interest is a wide range of research and professional interests in the STEM education field (UBC, 2020). In addition, Moeti, Mgawi & Waitshega (2017) and Ashour & Hawamdeh (2012) emphasize that postgraduates need to perceive critical thinking skills to be professional researchers or lecturers. The learning process conducted in Science Education major at the Universitas Negeri Surabaya postgraduate program focuses on developing critical thinking, creative thinking and problem solving, communication, and collaboration skills (S2pendsains, 2020). Physics is a way to understand our world and recognize the underlying principles and laws that connect to the disparate phenomena of our physical world, which is full of content and meaning (Rau, 2017). It is an authentic, essential, logical, and rational science that underlies the development of science and technology. In connection with its materials, problem-solving and decision-making skills can be carried out through some stages of a thinking process, which directly promote critical thinking skills and conceptual understanding of physics (Magno, 2010). Critical thinking requires students to be thorough, purposeful, and deliberate, focus on the main issue, and evaluate all parts of its complex and challenging claims and arguments (Sasson, Yehuda, & Malkinson, 2018). In a scientific context, the evaluation aspect involves interpreting data, drawing accurate conclusions from the data, comparing and evaluating models and data, estimating methods, and deciding how to proceed in an investigation (Walsh, Quinn, Wieman, & Holmes, 2019). In a science classroom, students' critical thinking can be identified by their ability to respond to daily life problems (Suhirman, Prayogi, & Asy‘ari, 2021). Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.43107 470 J.Sci.Learn.2022.5(3).469-477 Regardless of the education level, the learning process is currently required to adapt to the conditions of the Covid-19 pandemic. The Covid-19 outbreak was first detected in Wuhan City, China, in December 2019 and was declared a pandemic by WHO on March 11, 2020. In education, various countries have responded to adjusting the learning process according to the conditions of their respective countries. In Indonesia, the learning process is conducted from home, based on the Letter of the Minister of Education and Culture of the Republic of Indonesia Number 4 of 2020 concerning the Implementation of Education Policies in an Emergency for the Spread of Coronavirus Disease (Covid-19). In higher education, the home learning process is carried out online. Online lectures have been practiced for a long time, but during the Covid- 19 pandemic, this method has been compulsory in many countries. Kuntarto (2017) states that online learning is more effective and able to increase students’ mastery of subject material compared to the use of a face-to-face learning mode. The online lecture system positively contributes to encouraging disparities in the quality of higher education in Indonesia (Mustofa, Chodzirin, & Sayekti, 2019). Rosdiana, Widodo, Nurita & Fauziah (2018) state that learning vibrations and waves through online learning improves science teacher prospectives’ problem-solving and graphing skills. However, various obstacles arise when a learning process is carried out entirely online without a face-to-face presence for a long time. The most prominent obstacle is that learning activities are too boring for students. Livana, Mubin, & Basthomi (2020) state that 55.8% of students complained that online learning was boring. Cao et al. (2020) also convey that academic activity delays were positively related to student anxiety during the Covid-19 pandemic. The cause of students’ stress might involve academic problems, interpersonal problems, learning activities, social relationships, encouragement and desire, and group activity (Yusoff & Rahim, 2010). On the other hand, it was also found that, in general, assignments did not have a significant effect on student stress levels (Mardiati, Hidayatullah, & Aminoto, 2018). Therefore, the idea of learning to bridge the boredom of online lectures, appropriate assignments, and adequate results is necessary when the situation demands a complete online learning process. Therefore, one of the ideas applied in this study is an online flipped classroom. The flipped classroom is a learning approach that uses an online platform with flexible time (Karabulut-Ilgu, Cherrez, & Jahren, 2018). In its application, students complete several online activities to prepare the face-to- face learning on campus. It is dissimilar to the traditional lecture, where a professor must present new materials (Reidsema, Kavanagh, Hadgraft, & Smith (Eds), 2017). Instead, in a flipped classroom, instructors provide online materials for students to read, watch, outline, summarize, understand, and present on their own so that the classical method can be allocated to student-centered learning activities (Baytiyeh, 2017). Although studies on flipped classrooms are still required, most publications (e.g., peer- reviewed) describe it as a type of blended learning (Abeysekera & Dawson 2015). The application of flipped classrooms in lectures has several advantages, for example, offering flexibility, increasing interaction between lecturers and students as well as students and students, and developing professional skills (Karabulut-Ilgu, Cherrez, & Jahren, 2018). Baepler, Walker, & Driessen (2014) found that students who learned using flipped classroom model had better performance than those who knew with the traditional model. The application of flipped classrooms is considered appropriate because students can study literature or lecture assignments efficiently and flexibly. They also can discuss the materials or tasks face-to-face. Marlowe (2012) shows that flipped classrooms could reduce students’ stress levels and increase their semester scores. However, the nature of flipped classrooms cannot be applied in many countries due to the existing Covid-19 pandemic. In the flipped classroom, there is a face-to-face session. However, during the Covid-19 pandemic, many countries still do not allow face-to-face meetings. Therefore, the face-to-face session in flipped classrooms is replaced by face-to-face online sessions called Online Flipped Classroom. For example, Reidsema, Kavanagh, Hadgraft, & Smith (Eds), (2017) have not included face-to-face online ideas in the flipped classroom section in books on flipped classrooms for higher education. To the researchers’ knowledge, there is little research on an online flipped classroom. Weinhandl, Lavicza, & Houghton (2020) conducted a study about an online flipped classroom for mathematics class, which is different from the present study's focus. Smith (2020) also promoted online, face-to-face sessions that showed effective learning times and better student performance. Therefore, implementing a fully online flipped classroom with a duration of one semester in physics lectures to facilitate the development of critical thinking skills is worth researching. Thus, this study aims to determine whether the implementation of the online flipped classroom in School Physics lectures can develop science education postgraduate students' critical thinking skills. 2. METHOD The study was pre-experimental, using one group pretest-posttest design. This design in Table 1 let the participants be measured on their outcomes before and after the treatment (Mertens, 2010). The research subjects comprised 13 students who took the School Physics course for the academic year of 2020/2021 at the Science Education Postgraduate Program at one public university in East Java Province, Indonesia. Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.43107 471 J.Sci.Learn.2022.5(3).469-477 Regarding gender, the ratio of male and female students was 5:8. The treatment in this study was using an online flipped classroom with School Physics materials. The treatment encompassed several learning strategies: learning the material, conducting investigations, creating presentation files, carrying out an online face-to-face discussion, and accomplishing assignments. The response variable of the present study was critical thinking skills in School Physics, defined as the student's skills in explaining, making arguments, and assessing the truth shown by the critical thinking skills test score in School Physics. The researcher hypothesized that the treatment was effective, so the outcome scores improved. On the contrary, the treatment had no effect when the scores did not change. However, the researchers did not claim that the online flipped classroom was the best strategy among other strategies for online learning. The online flipped classroom strategy was implemented in the lecture process by changing lectures from face-to- face to online, according to the flow chart in Figure 1. In the online flipped classroom research, learning began with introductory sessions carried out synchronously. Based on the introduction, students conducted inquiries and searched for learning resources to make presentation files on certain topics according to the results of the distributed tasks. The lecturer gave feedback on the uploaded presentation files, and then the student corrected them according to the lecturer's suggestion. Afterward, face-to- face online learning was conducted where students presented their work while the other students responded. Lecturers provided reinforcement, clarification, and further suggestions if necessary. This online face-to-face was continued with the task of analyzing, making explanations, and initiating arguments related to potential misconceptions according to physics topics discussed asynchronously. These activities were carried out using Google Classroom Learning Management System (LMS). This LMS was chosen because it was easy, quickly accessible, and integrated with various hardware owned by students. Students conducted studies encompassing concepts, principles/laws along with their applications, and potential misconceptions on the topics, including 1) kinematics, 2) dynamics, 3) heat and thermodynamics, 4) waves, sound, and optics, 5) electricity, 6) magnetism and electromagnetic induction, and 7) earth and space. The inquiry activity in this study was an inquiry designed independently by students according to the topic using existing equipment and materials according to the context and virtual laboratories. Students were required to formulate problems, hypotheses, and methods to test hypotheses, present data, analyze data and draw conclusions. Thus, students were expected to develop their abilities in building arguments based on deductive and inductive reasoning. A rubric assessed this activity that two experts on science education had validated. The activities of lecturers and students were observed based on the activities at LMS by detailing the personalization o f lecturers and students in the pre-class and in-class phases based on Koh's (2019) idea, however, focusing on various activities. Critical thinking skills were measured through a critical thinking skills test in School Physics. The test instrument used consisted of 11 questions on critical thinking skills, of which the indicators of essential skills of thinking referred to Ennis (2011), namely: 1) making arguments equipped with mathematical models based on deductive analysis of physical symptoms; 2) making arguments equipped with a mathematical model based on an inductive analysis of physical symptoms; 3) explaining the statement of principles/laws of physics, and 4) assessing the truth of the statement accompanied by arguments based on the concepts/principles/laws of physics. The test questions consisted of a series of question formulation according to the indicators, internal reviews, revisions, and expert validation and revision. Pre-test questions differed from post-test ones; however, both are equivalent. In addition, science education experts validated the content, construction, and language use questions. Table 1 Experiment design O1 X O2 Initial measurement (pretest) of critical thinking skills related to School Physics The treatment given using an online flipped classroom in School Physics course The final measurement (posttest) of critical thinking skills related to School Physics Figure 1 The implementation of the online flipped classroom used in the present study ntroduction to ectures synchronous) Inquiry, information retrieval, and file presentation (asynchronous) Revision of the presented file based on lecturer’s feedback asynchronous) Presentation and discussion (synchronous) The further assignments: conducting analysis, making explanations, and initiating arguments related to potential misconceptions according to the topics discussed (asynchronous) Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.43107 472 J.Sci.Learn.2022.5(3).469-477 This study also used a questionnaire to reveal students’ responses to the School Physics learning process with an online flipped classroom strategy. This response was mandatory to obtain an overview of what science education graduate students experienced and felt regarding the learning process using the online flipped classroom strategy. In addition, the response was essential to get feedback that could be improved in the future. The questionnaires were administered online through the LMS. This study used descriptive and inferential data analysis techniques. Descriptive analysis was a description of increasing critical thinking skills using the normalized change (c) equation proposed by Marx & Cummings (2007) (see table 2). Sriyansyah & Azhari (2017) also suggested normalized change analysis compared to normalized gains. The inferential analysis was a median difference between the post-test and pre-test scores using nonparametric-inferential analysis of the Related-Samples Wilcoxon Signed Rank Test (Siebert & Siebert, 2018). This Table 2 Equations and criteria for normalized change Score conditions Equation Criteria c Value Category Postest < Pretest pre prepost c − = negative Decline Postest = pretest - 0 No improvement Postest > Pretest pre prepost c − − = 100 0 < c  0.3 Low 0.7 < c  0.7 Moderate 0.7 < c  1 High Table 3 Implementation of online filpped classroom Meeting Activities Lecturer Activities Student Activities 1 Explanation of the lesson plan in one semester ahead and pretest, how to learn, assignments, and assessments Providing information, relevant learning resources, and assignments Listening, reading, answering, conducting collaborative groups 2 Independent tasks: Inquiry with a virtual laboratory, collaboratively online, making reports Monitoring, providing assistance Working on assignments collaboratively, consulting 3 Synchronous discussion of the results of inquiry, assignment to conduct studies according to the selected topic Listening, providing feedback, giving follow-up assignments Conducting presentations and discussions, carrying out follow-up assignments 3-4 Independent assignments: students collaboratively conducted studies including concept maps, conceptual descriptions, principles/laws, applications, potential misconceptions, misconceptions remediation ideas, and inquiry ideas on an agreed topic. The results of the study were in the form of presentation file. Monitoring, providing assistance Providing feedback on the results of the study Working on assignments collaboratively, consulting and making revisions 5 Students made video presentations Monitoring, providing assistance. Working on assignments collaboratively, consulting 6-14 Presentation and discussion synchronously and asynchronously about the study results performed by each group Providing feedback, reinforcement, and clarification, if necessary, as well as giving the task of making explanations and arguments related to the conceptions of the topic Carrying out further tasks (applications in problem solving and building arguments), consulting 15 Wrap-up Summing up a review, carrying out reinforcement, and giving insight for the betterment Clarifying 16 Postest and questionnaire completion Sending posttest and questionnaire Answering Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.43107 473 J.Sci.Learn.2022.5(3).469-477 analysis was performed to determine whether the difference in pre-test and post-test scores was significant. Because of the small data, the researcher could not assume that the subjects were normally distributed and homogeneous, so a nonparametric analysis was undertaken. The significance level was .05. The inferential analysis was carried out using SPSS version 26.0. 3. RESULT AND DISCUSSION Table 3 shows the online flipped classroom implemented in the first semester of 2020/2021. Table 3 depicts that the online flipped classroom took place with students who conducted inquiries and studies first, followed by virtual face-to-face presentations and discussions. Then, the activity was continued with a follow- up assignment. This pattern followed the Flipped Classroom strategy according to Bergmann & Sams (2012), Reidsema, Kavanagh, Hadgraft, & Smith (Eds), (2017), and Baytiyeh (2017), but every learning stage was conducted online, named an online flipped classroom. Figure 2 shows an example of the online flipped classroom along with its example of the results of students’ collective assignments in LMS. Critical thinking aspects related to providing explanations, making deductive arguments, and making arguments inductively were developed through online inquiry activities in the form of investigating gas simulations in closed spaces. Table 4 portrays the results of the student inquiry report data analysis. Table 4 conveys that, on average, students obtained online inquiry results of 97 with a standard deviation of 4.9, which were in a very good category. The point that was emphasized during the online discussion was analyzing data. Some students took logical leaps in data analysis by deductively mentioning specific laws. Supposedly, in the data analysis phase, students formulated a mathematical model based on data trends (e.g., graphs). That was also to develop the ability to make arguments based on students' inductive reasoning. Table 5 shows the results of the descriptive analysis of the pre-test, post-test, and c values for students' critical thinking skills. Figure 2 Examples of online flipped classroom in LMS Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.43107 474 J.Sci.Learn.2022.5(3).469-477 Based on Table 5, it was found that there was an increase in students’ critical thinking skills, which was indicated by the elevating average pre-test score of 42 to 83 in the post- test. Such improvement was categorized in a high category, with the average normalized change (c) of 0.71. A difference test in pre-test and post-test scores was conducted to determine whether online flipped classroom learning significantly affected students' critical thinking skills (See Table 6). In addition, hypothesis testing of the median difference between pre-test and postest scores was carried out using Related-Samples Wilcoxon Signed Rank Test. The hypotheses were drawn as follows: Ho: there is no median difference between pre-test and post-test scores. Ha: there is a median difference between pre-test and post-test scores. Table 6 shows that Ho was rejected, and Ha was accepted. Thus, the post-test and pre-test scores for critical thinking were significantly different. These results reinforced the findings of He et al. (2017), who claim that flipped classrooms in architectural physics produced better students’ learning performances and abilities than the traditional classes. This study showed deeper learning in conducting investigations, browsing the literature, formulating presentation files, and discussing synchronously in the virtual face-to-face phase. The strength of the online flipped classroom to facilitate students to search various literature and conduct collaborative online investigations was also depicted by Sun, Wu & Lee (2017). Sun, Wu & Lee. (2017) argued that the flipped classroom model promoted a learning environment that prompted its learners to seek external Table 4 Inquiry results No Inquiry Aspect Mean Score Standard Deviation Interpretation Errors that occur 1 Formulating a problem 100 0 Very good - 2 Formulating a hypothesis 98 6.9 Very good Incomplete hypothesis 3 Designing a hypothesis testing steps 100 0 Very good - 4 Presenting data 100 0 Very good - 5 Analyzing data 88 13.0 Very good Inductive analysis mixed with deductive reasoning (not based on data) 6 Drawing conclusion 96 9.4 Very good Not all conclusions were supported by data Overall average 97 4.9 Very good Table 5 Descriptive analysis of students’ critical thinking skills No Pretest Posttest c Values Improvement Criteria 1 45 82 0.67 medium 2 44 85 0.73 high 3 38 86 0.78 high 4 41 79 0.64 medium 5 56 87 0.70 high 6 24 82 0.76 high 7 31 76 0.65 medium 8 48 82 0.65 medium 9 38 84 0.74 high 10 46 86 0.75 high 11 51 83 0.65 medium 12 38 81 0.70 medium 13 47 88 0.76 high Average 42 83 0.71 high Standard Deviation 8.43 3.31 0.05 - Table 6 Hypothesis test summary Null Hypothesis Test Sig. Decision The median of differences between Pretest and Postest equals 0. Related-Samples Wilcoxon Signed Rank Test .001 Reject the null hypothesis. Asymptotic significances are displayed. p < .05 Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.43107 475 J.Sci.Learn.2022.5(3).469-477 helps proactively. That was consistent with what one student said, "... I think it is exciting. Honestly, this is my first time with a system like this implemented. The lecture was unique and challenging, making me have to search various sources, question ideas in my mind, and discuss with friends… ”. From this statement, the online flipped classroom allowed students to self-regulate, foster self- efficacy, and increase their learning outcomes, of which Chen & Hwang's (2019) study also revealed similar findings. In addition, the lecturers believed that the constructive feedbacks were helpful for the students. "In my opinion, synchronous face-to-face lectures let me understand the material discussed easily because, when the discussion is over, the lecturer still provides the explanation and restrengthens any misconceptions. That is attractive to me". That means that this online flipped classroom provided opportunities for lecturers to provide feedback to reinforce students’ motivation (Tricomi & DePasque, 2016). Zainuddin, Haruna, Li, Zhang, & Chu (2019) emphasized that the flipped classroom yielded positive learning outcomes in students' learning activities, such as learning motivation and engagement, social interaction, and self-directed learning skills. Figure 3 shows Radar Score diagram based on critical thinking indicators Indicator 1: Making arguments supported by a mathematical model based on deductive analysis of the physical context; Indicator 2: Making arguments supported by a mathematical model based on an inductive analysis of the physical context; Indicator 3: Explaining the statement based on the principles/laws of science-physics; Indicator 4: Assessing the validity of statements that were strengthened by arguments Following Figure 3, each average score on indicators of critical thinking portrayed an increase. Normalized change (c) to arguments supported with a mathematical model based on a deductive and inductive analysis of physical phenomena was in a high category. That indicated that providing opportunities for students to carry out activities to formulate problems, generate hypotheses, conduct investigations to obtain data, analyze data, and conclude inquiry activities had an impact on the ability to build arguments accompanied by inductive mathematical models. This fact shows that implementing online learning with flipped classrooms could give students scientific investigation skills. These results were in line with a study by Tan, Yangco, & Que (2020), which revealed that online science learning with flipped classrooms could improve students' science process skills and conceptual understanding. Marshall & Kotska (2020) also said that online learning with flipped classrooms strengthened teachers' presence virtually in a class. In other words, when students worked on assignments monitored by lecturers online at LMS, students felt the presence of lecturers. Many relevant studies stated that deductive thinking skills required continuous practice (Eglington & Kang, 2018; Carriera, Amando, & Jacinto, 2020). This study portrayed that tracing learning resources, making presentation files, and discussing them facilitated the students a lot in training their critical thinking. Moreover, McLean, Attardi, Faden, & Goldszmidt (2016) showed that flipped classrooms produced better learning processes and academic performance (Romero-García, Buzón-García, & Touron, 2018). Although the online flipped classroom facilitated students’ learning activities, lecturers’ feedback, and increased critical thinking skills, students still preferred offline face-to-face meetings. Some students said, “... it is better face-to-face; eye fatigue; unstable network constraints; extravagant; although flexible, the combination with face to face is better; can't feel interpersonal relationship... ”. Therefore, in the context of online learning, it was still in line with Susanti, Hidayati, Anggreiny & Maputra, (2020), who claimed that 83.6% of students stated online learning in the last semester was not effective because of difficulty in understanding learning materials, neglected task, boring, and no teacher’s explanation. Troublesome signals, not optimal face-to-face learning, overload task, low economy, challenging to ask questions, and less attention to grades. Irawan, Dwisona & Lestari (2020) showed that students started to get bored during online learning after the first two weeks of the learning process. Although the online flipped classroom implementation showed that students wanted some offline, face-to-face activities (to feel good, not to get tired, and not to have the feeling of interpersonal relationships), they showed good learning performance. They produced better critical thinking skills in School Physics. Figure 3 Radar Score diagram based on critical thinking indicators 0 20 40 60 80 100 1 2 3 4 Pretest Postest Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.43107 476 J.Sci.Learn.2022.5(3).469-477 4. CONCLUSION The online flipped classroom can facilitate the development of science education postgraduate students’ critical thinking skills in the School Physics course. Based on the pre-test and post-test analysis, it is found that there is a significant median difference between the pre-test and post-test scores, of which the average normalized change of 0.71 is in a high category. Developing these skills through collaborative online inquiry activities could work on analyzing concepts, principles, potential misconceptions, and ideas to overcome the misconceptions, face-to-face online discussions, and follow-up assignments. At each stage, there are reinforcement and feedback. If online learning is still a must, for example, because of a pandemic, then an online flipped classroom is worth choosing. This study, finally, suggests that different materials and subjects in online flipped classrooms might need to be undertaken REFERENCES Abeysekera, L., & Dawson, P. (2015). Motivation and cognitive load in the flipped classroom: Definition, rationale and a call for research. Higher Education Research & Development, 34(1), 1–14. https://doi.org/10.1080/07294360.2014.934336. Ashour, R. Q. & Hawamdeh, M. F. (2012). Critical thinking skills among postgraduate university students in Jordan. Journal of Institutional Research South East Asia, 10(2), 16-28. Baepler, P., Walker, J.D., & Driessen, M. (2014). It's not about seat time: Blending, flipping, and efficiency in active learning classrooms. Computers & Education, 78, 227–236. https://psycnet.apa.org/doi/10.1016/j.compedu.2014.06.006. Bassham, G., Irwin, W., Nardone, H., & Wallace, J. M. (2013). Critical thinking: A student's introduction (5th ed.). New York: McGraw-Hill. Baytiyeh, H. (2017). The flipped classroom model: when technology enhances professional skills. The International Journal of Information and Learning Technology, 34(1), 51-62. https://doi.org/10.1108/IJILT-07-2016-0025. Bergmann, J. & Sams, A. (2012). Flip Your Classroom: Reach Every Student in Every Class Every Day, Washington DC: International Society for Technology in Education. Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J., & Zheng, J. (2020). The psychological impact of the covid-19 epidemic on college student in China. Psychiatri Research. 287, 112934. https://doi.org/10.1016/j.psychres.2020.112934. Carriera, S., Amando, N., & Jacinto, H. (2020). Venues for Analytical Reasoning Problems: How Children Produce Deductive Reasoning. Educ. Sci., 10. 169, 1-23. https://doi.org/10.3390/educsci10060169. Chen, P.Y. & Hwang, G.J. (2019). An IRS-facilitated collective issue- quest approach to enhancing students’ learning achievement, self- regulation and collective efficacy in flipped classrooms. British Journal of Educational Technology, 50(4), 1996–2013. https://doi.org/10.1111/bjet.12690. Costa, A., & Kallick, B. (2014). Dispositions: Reframing Teaching and Learning. Thousand Oaks, CA: Corwin Press. Eglington, L. G. & Kang, S. H. K. (2018). Retrieval Practice Benefits Deductive Inference Educational Psychology Review, 30(1), 215-228. https://doi.org/10.1007/s10648-016-9386-y. Ennis, R. H. (2011). The Nature of Critical Thinking: An Outline of Critical Thinking Dispositions and Abilities. University of Illinois. https://education.illinois.edu/docs/default-source/faculty- documents/robert- ennis/thenatureofcriticalthinking_51711_000.pdf. Irawan, A.W., Dwisona, & Lestari, M. (2020). Psychological Impacts of Students on Online Learning during the Pandemic COVID-19. Konseli, 07(1), 53-60. https://doi.org/10.24042/kons.v7i1.6389. Karabulut-Ilgu, Aliye, Cherrez, & N.J., Jahren, C.T. (2018). A systematic Review of Research on The Flipped Learning Method in Engineering Education. British Journal of Techology, 49(3), 398-411. https://doi.org/10.1111/bjet.12548. Koh, J. H. L. (2019). Four pedagogical dimensions for understanding flipped classroom practices in higher education: A systematic review. Educational Sciences: Theory and Practice, 19(4), 14 - 33. https://doi.org/10.12738/estp.2019.4.002 . Kraisuth, D., & Panjakajornsak, V. (2017). Thai engineer ASEAN readiness: A structural equation model analysis. Asia-Pacific Social Science Review, 16(3), 96-117. https://doi.org/10.1177/2158244017745346. Kuntarto, E. (2017). Keefektifan model pembelajaran daring dalam perkuliahan Bahasa Indonesia di perguruan tinggi [The effectiveness of the online learning model in Indonesian Language lectures at universities]. JILE, 3(1), 99-110. http://dx.doi.org/10.24235/ileal.v3i1.1820. Livana, P.H., Mubin, M.F. & Basthomi. Y. (2020). Tugas pembelajaran penyebab stress mahasiswa selama pandemi Covid-19 [Learning tasks cause student stress during the Covid-19 pandemic]. Jurnal Ilmu Keperawatan Jiwa, 3(2), 203-208. Magno, C. (2010). The role of metacognitive skills in developing critical thinking. Metacognition Learning 5, 137–156. https://doi.org/10.1007/s11409-010-9054-4. Mardiati, I., Hidayatullah, F. & Aminoto, C. (2018). Faktor eksternal tingkat stres mahasiswa keperawatan dalam adaptasi proses pembelajaran [External factors of nursing students' stress levels in adapting the learning process], 7th Univ. Res. Colloqium 2018 STIKES PKU Muhammadiyah Surakarta, pp. 172–181. Marlowe, C. (2012). The effect of the flipped classroom on student achievement and stress. Unpublished master’s thesis, Montana State University. Marshall, H.W. & Kostka, I. (2020). Fostering Teaching Presence through the Synchronous Online Flipped Learning Approach. The Electronic Journal for English as a Second Language, 24(2), 1-14. Marx, J. D. & Cummings, K. (2007). Normalized change. American Journal of Physics 75(1), 88-91. https://doi.org/10.1119/1.2372468. McLean, S., Attardi, S.M., Faden, L., & Goldszmidt, M. (2016). Flipped classrooms and student learning: not just surface gains. Adv Physiol Educ, 40(1), 47-55. https://doi.org/10.1152/advan.00098.2015. Mertens, D. M. (2010). Research and evaluation in education and psychology : integrating diversity with quantitative, qualitative, and mixed methods - 3rd ed. California: SAGE Publications. Moeti, B., Mgawi, R. K., & Waitshega. (2017). Critical Thinking among Post-Graduate Diploma in Education Students in Higher Education: Reality or Fuss? Journal of Education and Learning, 6(2), 13-24. https://doi.org/10.5539/jel.v6n2p13. Mustofa, M. I., Chodzirin, M., dan Sayekti, L. (2019). Formulasi model perkuliahan daring sebagai upaya menekan disparitas kualitas perguruan tinggi [The formulation of the online lecture model as an effort to reduce the disparity in the quality of higher education]. Walisongo Journal of Information Technology, 1(2), 151-160. https://doi.org/10.21580/wjit.2019.1.2.4067. OECD (2018). The future of education and skills Education 2030. http://www.oecd.org/education/2030/E2030%20Position%20P aper%20(05.04.2018).pdf. Rau, A. R. P. (2017). What is physics? The individual and the universal, and seeing past the noise. Physics Essays, 30(1), 60-69. https://doi.org/10.4006/0836-1398-30.1.60. Reidsema, C., Kavanagh, L., Hadgraft, R., & Smith, N (Eds). (2017). The Flipped Classroom. Singapore: Springer. Romero-García, C., Buzón-García, O., & Touron, J. (2018). The Flipped Learning Model in Online Education for Secondary Teachers, Journal of Technology and Science Education, 9(2), 109-121. https://doi.org/10.3926/jotse.435. https://doi.org/10.1080/07294360.2014.934336 https://psycnet.apa.org/doi/10.1016/j.compedu.2014.06.006 https://doi.org/10.1108/IJILT-07-2016-0025 https://doi.org/10.1016/j.psychres.2020.112934 https://doi.org/10.3390/educsci10060169 https://doi.org/10.1111/bjet.12690 https://doi.org/10.1007/s10648-016-9386-y https://education.illinois.edu/docs/default-source/faculty-documents/robert-ennis/thenatureofcriticalthinking_51711_000.pdf https://education.illinois.edu/docs/default-source/faculty-documents/robert-ennis/thenatureofcriticalthinking_51711_000.pdf https://education.illinois.edu/docs/default-source/faculty-documents/robert-ennis/thenatureofcriticalthinking_51711_000.pdf https://doi.org/10.24042/kons.v7i1.6389 https://doi.org/10.1111/bjet.12548 https://doi.org/10.12738/estp.2019.4.002 https://doi.org/10.1177%2F2158244017745346 http://dx.doi.org/10.24235/ileal.v3i1.1820 https://doi.org/10.1007/s11409-010-9054-4 https://doi.org/10.1119/1.2372468 https://doi.org/10.1152/advan.00098.2015 https://doi.org/10.5539/jel.v6n2p13 https://doi.org/10.21580/wjit.2019.1.2.4067 http://www.oecd.org/education/2030/E2030%20Position%20Paper%20(05.04.2018).pdf http://www.oecd.org/education/2030/E2030%20Position%20Paper%20(05.04.2018).pdf https://doi.org/10.4006/0836-1398-30.1.60 https://doi.org/10.3926/jotse.435 Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.43107 477 J.Sci.Learn.2022.5(3).469-477 Rosdiana, L., Widodo, W., Nurita, T., & Fauziah, A. (2018). E-learning task analysis making temporal evolution graphics on symptoms of waves and the ability to solve problems. International Conference on Science Education (ICoSEd). https://doi.org/10.1088/1742- 6596/1006/1/012025. S2pendsains. (2020, 12 20). Profil Singkat Magister Pendidikan Sains [Brief Profile of Master of Science Education]. http://s2pendsains.pasca.unesa.ac.id/post/profil-singkat. Sasson, I., Yehuda, I. & Malkinson N. (2018). Fostering the skills of critical thinking and question-posing in a project-based learning environment. Thinking Skills and Creativity, 29, 203-212. https://doi.org/10.1016/j.tsc.2018.08.001. Siebert, C. F. & Siebert, D. C. (2018). Data analysis with small samples and non-normal data: nonparametrics and other strategies. New York: Oxford University Press. Smith, K.D. (2020), "Is it face time or structure and accountability that matter? Moving from a flipped to a flipped/hybrid classroom", Journal of Applied Research in Higher Education, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JARHE-08-2019- 0229. Sriyansyah, S.P. & Azhari, D. (2017). Addressing an Undergraduate Research Issue about Normalized Change for Critical Thinking Test. JPII, 6(1), 138-144. DOI: https://doi.org/10.15294/jpii.v6i1.9602. Suhirman, Prayogi, S., & Asy‘ari, M. 2021). Problem-Based Learning with Character-Emphasis and Naturalist Intelligence: Examining Students Critical Thinking and Curiosity. International Journal of Instruction, 14(2), 217-232. https://doi.org/10.29333/iji.2021.14213a. Sun, J.C.Y., Wu, Y.T., & Lee, W. (2017). The effect of the flipped classroom approach to Open CourseWare instruction on students’ self-regulation. British Journal of Educational, 48(3), 713-729. https://doi.org/10.1111/bjet.12444. Susanti, M., Hidayati, I., Anggreiny, N., & Maputra, Y. (2020). School from home during COVID-19 pandemic, a descriptive study: effectivity of learning towards high school students in West Sumatra. International Conference of Psychology, KnE Social Sciences, 430– 445. https://doi.org/10.18502/kss.v4i15.8231. Tan, R. M., Yangco, R. T., & Que, E. N. 2020). Students’ conceptual understanding and science process skills in an inquiry-based flipped classroom environment. Malaysian Journal of Learning & Instruction, 17(1), 159-184. https://doi.org/10.32890/mjli2020.17.1.7 Tricomi, E. & DePasque, S. (2016), "The Role of Feedback in Learning and Motivation", Recent Developments in Neuroscience Research on Human Motivation (Advances in Motivation and Achievement, Vol. 19), Emerald Group Publishing Limited, Bingley, pp. 175-202. https://doi.org/10.1108/S0749-742320160000019015. UBC (2020, 12th 20). Master of Education in Science Education (MEd). Retrieved January 3 2021 from Graduate and Postdoctoral StudiesGraduate School: https://www.grad.ubc.ca/prospective- students/graduate-degree-programs/master-of-education-science- education. Walsh, C., Quinn, K.N., Wieman, C., & Holmes, N.G. (2019). Quantifying critical thinking: Development and validation of the physics lab inventory of critical thinking. Phys. Rev. Phys. Educ. Res., 15, 010135, 1-17. https://doi.org/10.1103/PhysRevPhysEducRes.15.010135. Weinhandl, R., Lavicza, Z., & Houghton, T. (2020). Designing online learning environments for flipped approaches in professional mathematics teacher development. Journal of Information Technology Education: Research. 19. 315-337. https://doi.org/10.28945/4573. Yusoff, M.S.B, & Rahim, A.F.A. (2010). The medical student stressor quistionnaire (MSSQ) manual. Kota Bharu: KKMED Publication. Zainuddin, Z., Haruna, H., Li, X., Zhang, Y. & Chu, S.K.W. (2019). A systematic review of flipped classroom empirical evidence from different fields: what are the gaps and future trends? On the Horizon, 27(2), 72-86. https://doi.org/10.1108/OTH-09-2018-0027. https://doi.org/10.1088/1742-6596/1006/1/012025 https://doi.org/10.1088/1742-6596/1006/1/012025 http://s2pendsains.pasca.unesa.ac.id/post/profil-singkat https://doi.org/10.1016/j.tsc.2018.08.001 https://www.emerald.com/insight/search?q=Kara%20Diane%20Smith https://www.emerald.com/insight/publication/issn/2050-7003 https://doi.org/10.1108/JARHE-08-2019-0229 https://doi.org/10.1108/JARHE-08-2019-0229 https://doi.org/10.15294/jpii.v6i1.9602 https://doi.org/10.1111/bjet.12444 https://doi.org/10.18502/kss.v4i15.8231 https://doi.org/10.32890/mjli2020.17.1.7 https://doi.org/10.1108/S0749-742320160000019015 https://doi.org/10.1103/PhysRevPhysEducRes.15.010135 https://doi.org/10.28945/4573 https://doi.org/10.1108/OTH-09-2018-0027