Indonesian Review of Physics (IRiP) p-ISSN: 2621-3761 | e-ISSN: 2621-2889 Vol.4, No.2, December 2021, pp. 61 - 69 DOI: 10.12928/irip.v4i2.3814 http://journal2.uad.ac.id/index.php/irip Email: irip@mpfis.uad.ac.id 61 Evaluation of STEM-Based Physics Learning on Students' Critical Thinking Skills: A Systematic Literature Review Winarti1*, Dwi Sulisworo2, Narentheren Kaliappen3 1 Department of Physics Education, Universitas Islam Negeri Sunan Kalijaga, Indonesia 2 Postgraduate Program of Physics Education, Universitas Ahmad Dahlan, Indonesia 3 School of International Studies, Universiti Utara Malaysia, Malaysia Email: winarti@uin-suka.ac.id Article Info ABSTRACT Article History Received: Jul 24, 2021 Revised: Dec 14, 2021 Accepted: Dec 27, 2021 This article discusses the importance of STEM learning in optimizing critical thinking skills in physics learning. It presents an overview of STEM learning to identify, train and improve students' critical thinking skills to assess misconceptions in the science education research literature since 2010. This study published articles in primary science education journals and indexed them in the leading research database to obtain data on STEM-based physics learning and critical thinking skills. To identify relevant studies in the literature, we conducted a systematic search of the two databases with the document analysis method. The investigation was limited to Indonesia's studies published between 2010 and 2020 for the most recent STEM-based physics learning studies and critical thinking skills based on the most recent findings. This study summarizes STEM learning at the high school and the undergraduate students. The implementation of the research related to STEM-based physics learning with critical thinking skills variables was in various teaching materials, learning models, and learning media. Teachers taught physics concepts using the STEM approach to improve critical thinking skills, including optical tools, thermodynamics, temperature and heat, dynamic fluids, and sound waves. The findings of this study can be used as a reference in the development of STEM learning patterns in the realm of science and physics education to improve students' critical thinking skills, which are the demands of skills in the 21st century that students must possess. This is an open-access article under the CC–BY-SA license. Keywords: Physics Learning STEM-based Learning Critical Thinking Skills Misconceptions in Science To cite this article: W. Winarti, D. Sulisworo, and N. Kaliappen, “Evaluation of STEM-Based Physics Learning on Students’ Critical Thinking Skills: A Systematic Literature Review,” Indones. Rev. Phys., vol. 4, no. 2, pp. 61–69, 2021, doi: 10.12928/irip.v4i2.3814. I. Introduction Many reviews on STEM (Science, Technology, Engineering, and Mathematics) arose in 1990 [1]. STEM in the curriculum focuses on the notion of educating learners in four integrated disciplines. They are science, technology, engineering, and mathematics with an inter- disciplinary and application approach [2]. STEM is a science discipline and curriculum with science, technology, engineering, and mathematics. The acronym STEM was introduced in 1990 by the science administrator of The National Science Foundation (NSF) of the United States of America [3]. Previously, the organization used the term STEM when referring to a particular discipline's career field or an integrated science- skill curriculum. In 2001, an NSF's biologist, educational, and human resource director assistant named Judith Ramaley in America re-arranged words and created an acronym of STEM [4]. Since the moment, any STEM- focused curriculum has widely spread outside of the United States of America. The curriculum has specifically developed programs based on the areas such as Asia, Australia, China, France, Southern Korea, Taiwan, and English [5]. The findings of international studies, such as TIMSS (Trends in International Mathematics and Science Study), are regularly researched on an international scale to compare mathematics competence and cognitive science achievement of fourth and eighth graders [6], [7]. On the http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& http://journal2.uad.ac.id/index.php/irip http://creativecommons.org/licenses/by-sa/4.0/ http://creativecommons.org/licenses/by-sa/4.0/ Indonesian Review of Physics (IRiP) Vol.4, No.2, December 2021, pp. 61 - 69 62 Winarti, et al. Evaluation of STEM-Based Physics Learning on Students’ …. p-ISSN: 2621-3761 e-ISSN: 2621-2889 other hand, PISA (Program for International Student Assessment) promotes a three-year assessment of the cognition and skills of learners aged 15 years [8], [9]. The findings obtained by PISA have also been carried out by several researchers who researched the implementation of the STEM learning approach among high school students. Many high school students are still stuck in participating in integrated learning simultaneously by prioritizing aspects of science, technology, engineering, and mathematics [10]. Moreover, each student has a dominant ability in only one aspect. However, this is not a significant problem in implementing STEM-based physics learning for high school students. In studying physical phenomena, students must master mathematical abilities [11]. In addition, students also need to adapt to technology and engineering to solve any existing physics problems. That is, STEM learning in physics can strengthen students' mathematical, technological, and engineering abilities [12]. It can also support the implementation of learning in other science groups such as chemistry and biology. Chemistry and biology learning in the future will certainly be related to each other with physics, which must prioritize understanding mathematical concepts to discuss these phenomena [13]. The STEM learning approach can optimize students' abilities in learning physical chemistry and physical biology by prioritizing strengthening mathematical abilities and mastering technology to help understanding and problem solving [14]. The STEM approach does not significantly impact if its implementation is carried out on arts and social- humanities learning whose scientific basis does not require students to understand mathematical abilities. However, the STEM learning pattern that is carried out by combining several disciplinary approaches can also be used to optimize students' abilities in learning arts and humanities if the implementation of learning is integrated with similar branches of science [15]. Thus, educators need to improve the implementation of learning that integrates several sciences or expertise so that students can grow various views in dealing with each problem. In physics learning, every educator and student needs to synergize in applying physics and mathematics learning simultaneously because mathematics cannot be separated from physics and as a step in implementing the STEM approach according to learning objectives [16]. Furthermore, some teachers integrate project-based activities that require cognitive and psychomotor implementation in certain areas, such as engineering. In some instances, extracurricular activities are included as team competitions. They can be added together or extended for a specific purpose. Undergraduate students also have the opportunity to spend their time with STEM experts regarding the availability of on-the-job training programs [17]–[20]. STEM has often been studied to measure, improve, and train students' critical thinking skills [21]–[26]. In addition, teachers can apply STEM- based learning to train, improve, and measure students' creative thinking skills at various levels [27]–[30]. Integrated STEM learning can improve critical thinking skills [31]–[35]. This finding is also by the findings in physics learning which shows that physics learning carried out by integrating the traditional game-based STEM approach can improve students' critical thinking skills [36]. This can happen considering that physics learning is fun for students. Students feel enjoy during learning, and they actively question every physical phenomenon that occurs in real or virtual traditional games through simulations [37]. Simultaneously integrating science, technology, engineering, and mathematics in physics learning raises student curiosity in responding to every physical phenomenon from various points of view and is properly accommodated [38]. Students often question every narrow gap from every discussion of physics material delivered by the teacher. This is one of the indicators that physics learning is by the learning objectives and can grow students' critical thinking skills [39]. However, some findings show that the STEM approach does not really contribute to optimizing students' critical thinking skills [40]. Thus, the success factor in optimizing students' critical thinking skills through STEM learning is not only from the draft learning guidelines that have been compiled. However, what is far more important is the teacher's ability to identify student needs, classroom mastery skills, physics and STEM concepts mastery skills, and problem- solving skills by integrating various STEM concepts. This review of contrasting findings is very important in STEM- based learning and the correlation of critical thinking skills in high school physics learning. This study chose critical thinking skills as a variable because STEM learning is strongly correlated with skills to address each problem in- depth and accurately. Moreover, mastery of critical thinking skills by mastering technological advances, engineering, science, and mathematics is very much needed in the development of the 21st century [41], [42]. In its correlation with the critical thinking skill of physics lessons, STEM has specific features. Physics learning is strongly correlated with physics concepts with its mathematics language as the mean [41]-[43]. Learners would understand the contextual process of physics concepts around their surroundings [44]–[46]. For example, learners that listened to accident news on the road. They might explain the accident using momentum and impulse concepts between two collided objects. Another example could be a crashed object. Teachers might apply mathematics calculation to identify which one was the victim. In the academic setting, students could create reconstructions optimally. Thus, it is possible to improve learners' physics understanding and their learning achievements. Furthermore, this study aims to examine the importance of STEM learning to optimize students' critical thinking skills in learning physics. This article also presents an overview of STEM learning to identify, train, and improve students' critical thinking skills to comparatively assess misconceptions in the 2010s science http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& Indonesian Review of Physics (IRiP) Vol.4, No.2, December 2021, pp. 61 - 69 63 Winarti, et al. Evaluation of STEM-Based Physics Learning on Students’ …. p-ISSN: 2621-3761 e-ISSN: 2621-2889 education research literature. The significance of the research relates to its contribution to the literature by describing STEM learning and critical thinking skills, especially in physics learning in Indonesia. In addition, this study also briefly compares STEM learning and critical thinking skills learning in general by considering Indonesia's strengths and weaknesses. Researchers apply STEM correlation analysis and critical thinking skills to enrich the literature on STEM-based physics learning, focusing on achieving student competencies in a comprehensive, focused, and based on actual needs. II. Theory STEM STEM learning combines elements of science, technology, engineering, and mathematics simultaneously in learning at the basic education level to higher education [47]. The implementation of STEM learning is mostly done in the science family, such as physics, chemistry, biology, and science. Although it is undeniable that the STEM approach can also be carried out in social humanities learning if it still integrates the relationship between learning objects and real problems that occur in students' daily lives [48]. Meanwhile, the STEM approach is a learning approach that can teach two or more STEM subjects related to holistic and comprehensive learning activities that can optimize students' critical thinking skills [49]. Therefore, STEM is a learning approach that is often used to optimize some students' abilities so that they can be more adaptive to the problems that occur that must be faced from various perspectives. STEM education that is implemented in physics learning can make students more concerned about the physics material they are learning, which has an important role in solving every problem that occurs in everyday life [50]. Implementing STEM learning certainly requires effective strategies and patterns so that learning and students go hand in hand and foster meaningful learning for students [51]. Furthermore, creating a strategic approach to integrating STEM learning in physics requires resources, analysis of student needs, and a strong conceptual understanding of student characteristics, physics material, and STEM learning patterns [52]. STEM-based education could develop meaningful learning via systematic and integrated knowledge, concept, and skill [53]. It could also improve scientific literacy, motivation, investigation skill, experience, and critical thinking skills. STEM emphasizes all aspects of learning. STEM practice involves posting questions and defining problems. Develop and use planning models and activities to promote research. Analyze and interpret data with mathematics, information, and computer technology. Interpret and design solutions, including evidence-based arguments, arrange conclusions, evaluate, and communicate the process [54]. Meanwhile, the purpose of STEM is to develop skills for students to be applied in various situations and various problems of everyday life. STEM-based learning aims to create students with a balance of hard skills and soft skills. Critical Thinking Skills Several findings indicate that critical thinking skills have a relationship with problem-solving abilities. The findings state that problem-solving skills require students to think critically [55]. Learners with critical thinking skills can actively solve problems [56]. Realizing critical thinking skills in learning demands to explain, generalize, conclude deductively, and determine an accurate way. Demands of students to solve a problem. Demands students to understand how to find ideas, recognize the need for new knowledge, and determine possible steps applied to solving data collection and knowledge learning problems. Requirements to promote fair and objective judgment [57]. Critical thinking skills allow learners to analyze information, post arguments and supportive evidence, think widely with posting hypotheses, investigate to prove a phenomenon, and obtain new information [58]. Furthermore, optimizing students' critical thinking skills in learning can help students understand themselves and understand their ability to think deeply about the narrow gaps of a problem or phenomenon that occurs [59]. Students who are accustomed to always prioritizing critical thinking skills towards any phenomena or problems they face will become individuals who do not easily believe in news or information that has no clear evidence of clarity [60]. Optimizing critical thinking skills in students is influenced by several things such as learning styles, implementation of learning, ability to understand concepts, and problem-solving abilities [61]. Therefore, it can be concluded that critical thinking skills are deep- thinking skills that can question and contradict all kinds of narrow gaps from every existing problem so that new insights that are full of accuracy emerge. III. Method In this case, the investigation took articles published in the science-education journal and indexed journal outlets. It was to obtain the data of STEM-based and critical thinking skill learning on physics. The researchers applied systematic searches from two databases (Google Scholar and Science Direct) with a document analysis method to identify relevant literature studies. The two databases contain various scientific articles published by various reputable journals, especially scientific articles from Indonesian researchers. In addition, the keywords that we used during the search for scientific articles were the STEM approach in learning physics and the effect of STEM on critical thinking skills in learning physics. The limitation search was on Indonesian language studies published between 2010 until 2020. It was to get physics learning studies based on the latest STEM and critical thinking skills. The multistage process was followed by reading each article. All information from the reports was identified and discussed with the experts. After placing http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& Indonesian Review of Physics (IRiP) Vol.4, No.2, December 2021, pp. 61 - 69 64 Winarti, et al. Evaluation of STEM-Based Physics Learning on Students’ …. p-ISSN: 2621-3761 e-ISSN: 2621-2889 1940 articles with the applied keyword, abstract, only 116 articles revealed their focuses on STEM in physics learning, the correlation to critical thinking skills, and the given treatment. There was an iterative process during the article searches. Each obtained article reference list was added as a new reference source. The accepted articles were comprehensively investigated within physics learning based on STEM and critical thinking skills. A general discussion about STEM and critical thinking skills compares the articles' strengths and weaknesses. Figure 1 illustrates the study flow of the searching and selecting processes. Figure 1. Research flow IV. Results and Discussion The researchers grouped the research article percentage based on the population levels and the learners' samples to determine the distribution of STEM-based physics learning and its correlation toward critical thinking skills. Considerations for selecting the population and sample in this study are the results of research that has been published in various national journals using populations and samples from high school students during physics learning using the STEM approach with the main aim of optimizing their critical thinking skills. Furthermore, Table 1 shows the percentage of research results in STEM- based physics learning and critical thinking skills of Indonesian students. Table 1. The percentage of the STEM-based physics learning research and critical thinking skills of Indonesian students No. Education Level Numbers Percentage (%) 1 Senior High School 71 61 2 Undergraduate 45 39 Table 1 shows that teachers have promoted STEM- based physics and critical thinking skill learning in Indonesia in senior high schools (61%). It is higher than the university level, precisely the undergraduate degree (39%). Physics lessons included a foundation of science and biology at the junior high school level. Therefore, it was excluded from this research category. Physics lessons had thematic learning and primary science and other studies discussions at the primary school level. It was excluded for this STEM level category on this research review (it did not meet selected articles' criteria). Physics learning at senior high school or university levels had been an integral part of a lesson or a course. It showed that teachers focused physics learning at the university level on specific and comprehensive physics discussions. There were widely many studies about STEM at this education level. Unconsciously, STEM had existed in physics learning at university levels, such as fundamental physics (science), instrumentation (technology), laboratory praxis (engineering), and calculus (mathematics). At Senior High School levels, physics learning was still following conservative curriculum patterns. The changes are due to the government policy revitalization. It promoted comprehensive research on STEM-based physics and critical thinking skill learning at the senior high school level than the university level. Similar-themed article findings from other countries supported Indonesia's article review findings [3], [10], [12]. Table 2 shows that STEM-based physics and critical thinking skill learning in Indonesia were mostly applied to learning models to teach physics (59%). The following result is STEM-approach implementation on physics learning media based on STEM with a percentage of 24% and the performance on STEM-based physics teaching material with 17%. The STEM approach implementation on the learning model could be purely applied or in an integrated manner. For example, the integrated applications of STEM were Project-based learning, problem-based learning, and 5-cycle of learning. The STEM approach component implementations on learning materials were such as learners' worksheets. They were reflected on the lesson plan arrangement and observable during learning syntaxes. The STEM approach implementation on learning media consisted of visual media (PowerPoint) and audiovisual media (video). Table 2. The Implementations of STEM and Critical Thinking Skill Approaches No. The implementations on the learning Numbers Percentage (%) 1 STEM-based Physics Learning Materials 20 17 2 The Model of STEM- based Physics Learning 68 59 3 STEM-based Physics Learning Media 28 24 Table 2 describe STEM approach implementation that could be the basis of developing media, models, and learning material. The most frequent performance was on learning model development from one hundred sixteen http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& Indonesian Review of Physics (IRiP) Vol.4, No.2, December 2021, pp. 61 - 69 65 Winarti, et al. Evaluation of STEM-Based Physics Learning on Students’ …. p-ISSN: 2621-3761 e-ISSN: 2621-2889 articles. The STEM-based physics learning development could be a specific syntax (as the pure STEM approach and syntax). On the other hand, it could be integrated with other existing models proven effective for learning physics. One of the researches was the integrated STEM- based learning model with project-based learning model. Teachers facilitated learners to design and realize the projects based on the teachers' already taught physics concepts as facilitators. The learners should follow the STEM-based learning syntax with the PjBL model. It made learners understood the essence of physics and the mathematics language instead of a mere physics concept and vice versa. Learners who could work on mathematics-physics problems would be more able to analyze the contextual phenomena physically. Their trained critical thinking skills were based on STEM syntax [62]. Investigations on STEM-based physics learning and its PjBL integration to improve critical thinking skills in other countries supported this finding. Han et al. [63] dan Verma et al. [64] found that learners could be better motivated with PjBL- STEM than those high-achievement learners with low critical thinking skills and achievements. A longitudinal study would allow researchers to get more data about the influences of STEM-PjBL integration on learners' critical thinking skills. Although teachers, as the facilitators, could not control each factor, the influential critical factor had significant influences on learners' learning achievements based on the improved critical thinking skills. Various articles cited STEM-based physics learning varied in terms of the levels, implementations, and physical concepts. In the first article in Table 3, the researchers developed the learner worksheet with the STEM approach on the learning flow and syntax to improve learners' critical thinking skills. The worksheet results based on the STEM approach proved that it could improve the learners' critical thinking skills. The same results were found in the second until the fifth research shown in Table 3. The STEM approach could influence the critical thinking skill of the learners. The findings of STEM-based physics learning, summarized in Table 3, had the same objective. It was to improve the learners' critical thinking skills. However, they might have different implementations, such as the learners' worksheets, the integrated project-based learning and STEM, and STEM assisted by Thinking Map media on the guided inquiry learning model. Table 3 shows the research varieties about STEM- based physics learning on physics concepts, such as optical instruments, sound waves, dynamic fluids, thermodynamics, temperatures, and heat. The first reference was the worksheet development based on the STEM approach [65]. The developed worksheet could improve learners' critical thinking skills. The second and third references showed similar things. Implementing STEM-based learning could improve learners' critical thinking skills [15], [66]. The fourth research developed project-based learning with a STEM approach. Before the intervention, the learners' critical thinking skills were categorized low. However, after being intervened, the learners' critical thinking skills improved into a moderate category [62]. The fifth reference also obtained the same thing. The STEM-based physics learning development assisted by Thinking Map media during a guided-inquiry model could improve the learners' critical thinking skills [67]. The findings of the articles in Table 3 represent the findings of STEM-based physics learning studies on physics concepts, such as temperatures and heat, thermodynamics, sound waves, dynamics fluids, and optical instruments. It showed that reviews about the theme had a diverse nature. STEM learning was proven to have a better implication on physics learning's critical thinking skills in Indonesia. It was in line with several types of research in other countries. They also summarized the STEM learning for physics lesson [68]–[72]. STEM- based learning could improve learners’ critical thinking skills. Table 3. STEM-based Physics Learning Reference to Improve Critical Thinking Skills No. Physical Concept Research Findings Reference 1 Optical Instrument The learners’ worksheets with the STEM approach could improve critical thinking about optical instruments. (Lestari, Astuti, & Darsono, 2018) 2 Sound waves Implementing the STEM learning approach could improve learners’ critical thinking skills. (Khoiriyah et al., 2018) 3 Dynamic Fluids The findings showed the influence of STEM learning on conceptual masteries and learners' critical thinking skills about dynamic fluid. (Rivai, Yuliati, & Parno, 2018) 4 Thermo- dynamics The results showed moderate improvement category of scientific- creative and critical thinking skills after being taught with project-based learning and STEM approach. (Cholisoh, 2019) 5 Tempera- ture and Heat STEM assisted by thinking maps on a guided-inquiry learning model significantly influenced the learners’ critical thinking skills about the materials. (Wastiti & Sulur, 2020) The syntax of STEM-based physics learning is as follows. 1) planning and promoting class sessions by involving all learners actively, no matter how large the class was; 2) using the technology properly during face-to- http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& Indonesian Review of Physics (IRiP) Vol.4, No.2, December 2021, pp. 61 - 69 66 Winarti, et al. Evaluation of STEM-Based Physics Learning on Students’ …. p-ISSN: 2621-3761 e-ISSN: 2621-2889 face, online, hybrid, and blended courses; 3) assessing the cognitive, psychomotor, and conceptual understanding performances of learners to learn the lessons, 4) facilitating learners to develop excellent problem-solving and communication skills, to develop creative thinking, critical thinking, cooperation, and autonomous learning; and 5) providing the STEM learners with various attributes and broader background. Educating learners with STEM lessons (if the teacher taught correctly) could prepare them to engage with their future lives despite their preferred profession later. The findings could show learners the importance of critical thinking and creative thinking skills to solve problems. The skills are also important in their lives when they encounter challenges. The researcher also suggests applying STEM-based contextual learning to solve 21st- century challenges. Thus, integrating STEM and critical thinking skills are important to create meaningful and joyful learning. V. Conclusion The multistage process of this research had eliminated 1940 articles about STEM-based learning. They were initially identified based on the title, abstract, and keyword. At the end of the article selection based on the whole content, relevance, range of years - from 2010 until 2020, limitation on physics learning, and Indonesian language articles. The research consisted of 116 items that the abstracts showed research focuses on STEM in physics learning and its correlation to critical thinking skills and the interventions. The findings showed the STEM implementation at senior high school and the undergraduate levels, especially in Indonesia. Besides that, any related research about STEM-based physics with critical thinking skills showed the performance of learning materials, learning models, and various learning media. Teachers applied the physics concepts using a STEM approach to improve the critical thinking skills on optical instruments, thermodynamics, heat and temperatures, dynamic fluids, and sound waves. This study still has several limitations, such as the main reference source used as the primary source of research is still limited from research results from Indonesia. The research results from Indonesia used in this article are also not entirely, or the majority have not been from reputable international journals. Therefore, further research is needed that can overcome the problems of this research by increasing the variety of research results in terms of geography, such as comparisons with countries in the Southeast Asian region or Asian countries. Further researchers also need to use reference sources from scientific articles that have been published in various reputable international journals so that their usefulness can be applied universally. VI. Acknowledgment Universitas Islam Negeri Sunan Kalijaga, Universitas Ahmad Dahlan, and Universiti Utara Malaysia who facilitate the access of all reports. References [1] N. R. Council, “Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics,” Washington, DC, 2011. [2] N. Khairiyah, Pendekatan science, technology, engineering dan mathematics (STEM). Bogor: Guepedia Publisher, 2019. [3] R. W. Bybee, The case for STEM education: Challenges and opportunities. Arlington: NSTA Press, 2013. [4] I. R. M. Association, STEM Education: Concepts, Methodologies, Tools, and Applications. Pennsylvania: IGI Global, 2014. https://doi.org/10.4018/978-1-4666- 7363-2 [5] R. M. Felder and R. Brent, Teaching and Learning STEM: A Practical Guide, San Franscisco: Jossey-Bass, 2016. https://doi.org/10.1063/PT.3.3556 [6] A. E. Beaton, “Mathematics Achievement in the Middle School Years. IEA’s Third International Mathematics and Science Study (TIMSS),” Rockville, 1996. [7] L. Wößmann, “The Effect Of Heterogeneity Of Central Examinations: Evidence From TIMSS, TIMSS-Repeat and PISA,” Educ. Econ., vol. 13, no. 2, pp. 143–169, 2005. https://doi.org/10.1080/09645290500031165 [8] OECD, “Learning for tomorrow’s world: First results from PISA 2003,” Paris, 2004. [9] OECD, “PISA 2012 results: Creative Problem Solving Students’ Skills in Tackling Real-Life Problems,” Paris, 2014. https://doi.org/10.1787/9789264208070-en [10] H. E. Chu, S. N. Martin, and J. Park, "A theoretical Framework for Developing an Intercultural STEAM Program for Australian and Korean Students to Enhance Science Teaching and Learning," Int. J. Sci. Math. Edu., vol. 17, no. 7, pp. 1251-1266, 2019. https://doi.org/10.1007/s10763-018-9922-y [11] J. M. G. Rodriguez, S. Santos-Diaz, K. Bain, and M. H. Towns, "Using Symbolic and Graphical Forms to Analyze Students’ Mathematical Reasoning in Chemical Kinetics," J. Chem. Edu., vol. 95, no. 12, pp. 2114-2125, 2018. https://doi.org/10.1021/acs.jchemed.8b00584 [12] P. Dorouka, S. Papadakis, and M. Kalogiannakis, "The Contribution of the Health Crisis to Young Children’s Nano-Literacy Through STEAM Education," Hellenic J. STEM Edu., vol. 2, no. 1, pp. 1-7, 2021. https://doi.org/10.51724/hjstemed.v2i1.18 [13] R. Hoffmann and J. P. Malrieu, "Simulation vs. Understanding: A tension, in Quantum Chemistry and Beyond. Part A. Stage setting," Angewandte Chem. Int. Ed., vol. 59, no. 31, pp. 12590-12610, 2020. https://doi.org/10.1002/anie.201902527 [14] A. Nurwahyunani, "Literature Review: a STEM Approach to Improving the Quality of Science Learning in Indonesia," J. Edu. Gift. Young Sci., vol. 9, no. 1, pp. 11- 17, 2021. https://doi.org/10.17478/jegys.853203 [15] L. V. Shukshina, L. A. Gegel, M. A. Erofeeva, I. D. Levina, U. Y. Chugaeva, and O. D. Nikitin, "STEM and STEAM Education in Russian Education: Conceptual Framework," Eurasia J. Math. Sci. Tech. Edu., vol. 17, no. 10, pp. 234-239, 2021. https://doi.org/10.29333/ejmste/11184 [16] A. Struyf, H. De Loof, J. Boeve-de Pauw, and P. Van http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& http://doi.org/10.1063/PT.3.3556 https://doi.org/10.1080/09645290500031165 https://doi.org/10.1787/9789264208070-en https://doi.org/10.1007/s10763-018-9922-y https://doi.org/10.51724/hjstemed.v2i1.18 https://doi.org/10.1002/anie.201902527 https://doi.org/10.17478/jegys.853203 https://doi.org/10.29333/ejmste/11184 Indonesian Review of Physics (IRiP) Vol.4, No.2, December 2021, pp. 61 - 69 67 Winarti, et al. Evaluation of STEM-Based Physics Learning on Students’ …. p-ISSN: 2621-3761 e-ISSN: 2621-2889 Petegem, "Students’ Engagement in Different STEM Learning Environments: Integrated STEM Education as Promising Practice?," Int. J. Sci. Edu., vol. 41, no. 10, pp. 1387-1407, 2019. https://doi.org/10.1080/09500693.2019.1607983 [17] B. Stein, A. Haynes, M. Redding, T. Ennis, and M. Cecil, “Assessing Critical Thinking in STEM and Beyond,” in Innov. E-learn. Instr. Tech. Assess. Eng. Edu., Dordrecht: Springer, 2007, pp. 79–82. https://doi.org/10.1007/978-1- 4020-6262-9_14 [18] M. Duran and S. Sendag, “A Preliminary Investigation Into Critical Thinking Skills of Urban High School Students: Role of an IT/STEM program,” J. Creat. Educ., vol. 3, no. 2, pp. 241–250, 2012. https://doi.org/10.4236/ce.2012.32038 [19] A. J. Gottesman and S. G. Hoskins, “CREATE Cornerstone: Introduction to Scientific Thinking, a New Course for STEM-interested Freshmen, Demystifies Scientific Thinking Through Analysis Scientific Literature,” CBE Life Sci. Educ., vol. 12, no. 1, pp. 59–72, 2013. https://doi.org/10.1187/cbe.12-11-0201 [20] K. L. Ramsey and B. Baethe, “The Keys to Future STEM Careers: Basic Skills, Critical Thinking, and Ethics,” Delta Kappa Gamma Bull., Austin, 2013. [21] S. F. Putri and E. Istiyono, “The Development of Performance Assessment of STEM Based Critical Thinking Skill in the High School Physics Lessons,” Int. J. Environ. Sci. Educ., vol. 12, no. 5, pp. 1269–1281, 2017. [22] N. Khoiriyah, A. Abdurrahman, and I. Wahyudi, “Implementasi Pendekatan Pembelajaran STEM untuk Meningkatkan Kemampuan Berpikir Kritis Siswa SMA Pada Materi Gelombang Bunyi [Implementation of STEM Learning Approach to Improve Critical Thinking Ability of High School Students on Sound Wave Material],” J. Ris. dan Kaji. Pendidik. Fis., vol. 5, no. 2, pp. 53–62, 2018. http://dx.doi.org/10.12928/jrkpf.v5i2.9977 [23] A. Tiara, “Pengaruh Pendekatan STEM (Science,Technology, Engineering, and Mathematic) Terhadap Keterampilan Berpikir Kritis dan Sikap Ilmiah Siswa Kelas XI pada Mata Pelajaran Biologi di MAN 2 Bandar Lampung [The Effect of STEM Approach (Science, Technology, Engineering, and Mathematics) on Critical Thinking Skills and Scientific Attitudes of Class XI Students in Biology Subjects at MAN 2 Bandar Lampung],” UIN Raden Intan Lampung, 2019. [24] N. Hidayati, F. Irmawati, and T. A. Prayitno, “Peningkatan Keterampilan Berpikir Kritis Mahasiswa Biologi Melalui Multimedia STEM Education [Improving Critical Thinking Skills of Biology Students Through Multimedia STEM Education],” JPBIO (Jurnal Pendidik. Biol., vol. 4, no. 2, pp. 84–92, 2019. http://dx.doi.org/10.31932/jpbio.v4i2.536 [25] S. H. Santoso and M. Mosik, “Keefektifan LKS berbasis STEM (Science, Technology, Engineering and Mathematic) untuk Melatih Keterampilan Berpikir Kritis Siswa pada Pembelajaran Fisika SMA [The effectiveness of STEM-based worksheets (Science, Technology, Engineering and Mathematics) to Train Students' Critical Thinking Skills in High School Physics Learning],” Unnes Phys. Educ. J., vol. 8, no. 3, pp. 248–253, 2019. https://doi.org/10.15294/upej.v8i3.35622 [26] Y. Kurniasih, G. Hamdu, and D. A. M. Lidinillah, “Asesmen Kinerja Berpikir Kritis pada Pembelajaran STEM dengan Media Lightning Tamiya Car [Assessment of Critical Thinking Performance in STEM Learning with Lightning Tamiya Car Media],” J. Ilm. Sekol. Dasar, vol. 4, no. 2, p. 174, 2020. http://dx.doi.org/10.23887/jisd.v4i2.25172 [27] I. Permana, “Pengaruh Pendekatan Science, Technology, Engineering, and Mathematics (STEM) Terhadap Peningkatan Keterampilan Berpikir Kritis Menggunakan Pembelajaran PBL pada Siswa Kelas X Materi Hukum Newton Tentang Gerak [The Effect of the Science, Technology, Engineering, and Mathematics (STEM) Approach on Improving Critical Thinking Skills Using PBL Learning in Class X Students of Newton's Law Of Motion],” Universitas Lampung, 2020. [28] H. R. Dewi, T. Mayasari, and H. Jeffry, “Peningkatan Ketrampilan Berfikir Kreatif Siswa Melalui Penerapan Inkuiri Terbimbing Berbasis STEM [Improving Students' Creative Thinking Skills Through the Application of STEM-Based Guided Inquiry],” in Sem. Nas. Pend. Fis. vol. 3, 2017, pp. 47–53. [29] K. D. Kristiani, T. Mayasari, and E. Kurniadi, “Pengaruh Pembelajaran STEM-PjBL Terhadap Keterampilan Berpikir Kreatif [The Effect of STEM-PjBL Learning on Creative Thinking Skills],” in Pros. Sem. Nas. Pend. Fis., 2017, pp. 266–274. [30] R. S. Pertiwi, “Pengembangan Lembar Kerja Siswa dengan Pendekatan STEM untuk Melatih Keterampilan Berpikir Kreatif Siswa pada Materi Fluida Statis [Development of Student Worksheets Using a STEM Approach to Train Students' Creative Thinking Skills on Static Fluids],” Universitas Lampung, 2017. [31] E. Triastuti, “Pembelajaran Berbasis STEM pada Materi Sel Volta untuk Meningkatkan Keterampilan Berfikir Kreatif Siswa [STEM-Based Learning on Voltaic Cells to Improve Students' Creative Thinking Skills],” J. Adi Karsa Teknol. Komun. Pendidik., vol. 15, no. 16, 2019. [32] Q. A’yun, A. Rusilowati, and L. Lisdiana, “Improving Students’ Critical Thinking Through STEM Digital Book,” J. Innov. Sci. Educ., vol. 10, no. 37, pp. 237–243, 2019. https://doi.org/10.15294/JISE.V8I3.35260 [33] K. L. Nelson, C. M. Rauter, and C. E. Cutucache, “Life Science Undergraduate Mentors in NE STEM 4U Significantly Outperform Their Peers in Critical Thinking Skills,” CBE Life Sci. Educ., vol. 17, no. 4, pp. 1–7, 2018. https://doi.org/10.1187/cbe.18-03-0038 [34] P. Onsee and P. Nuangchalerm, “Developing Critical Thinking of Grade 10 Students Through Inquiry-Based STEM Learning,” J. Penelit. dan Pembelajaran IPA, vol. 5, no. 2, p. 132, 2019. http://dx.doi.org/10.30870/jppi.v5i2.5486 [35] Widya, R. Rifandi, and Y. Laila Rahmi, “STEM Education to Fulfil the 21st Century Demand: A Literature Review,” J. Phys. Conf. Ser., vol. 1317, no. 1, 2019. https://doi.org/10.1088/1742-6596/1317/1/012208 [36] P. Y. Chen, G. J. Hwang, S. Y. Yeh, Y. T. Chen, T. W. Chen, and C. H. Chien, "Three decades of game-based learning in science and mathematics education: an integrated bibliometric analysis and systematic review," J. Comp. Edu., vol. 2, no. 1, pp. 1-22, 2021. https://doi.org/10.1007/s40692-021-00210-y [37] A. Bodzin, R. A. Junior, T. Hammond, and D. Anastasio, "Investigating engagement and flow with a placed-based immersive virtual reality game," J. Sci. Edu. Tech., vol. 30, no. 3, pp. 347-360, 2021. https://doi.org/10.1007/s10956-020-09870-4 [38] A. C. Graesser, J. P. Sabatini, and H. Li, "Educational psychology is evolving to accommodate technology, multiple disciplines, and twenty-first-century skills," Ann. http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& https://doi.org/10.1080/09500693.2019.1607983 http://dx.doi.org/10.1007/978-1-4020-6262-9_14 http://dx.doi.org/10.1007/978-1-4020-6262-9_14 http://dx.doi.org/10.4236/ce.2012.32038 https://doi.org/10.1187/cbe.12-11-0201 http://dx.doi.org/10.12928/jrkpf.v5i2.9977 https://doi.org/10.15294/upej.v8i3.35622 http://dx.doi.org/10.23887/jisd.v4i2.25172 https://doi.org/10.15294/jise.v8i3.35260 https://dx.doi.org/10.1187%2Fcbe.18-03-0038 http://dx.doi.org/10.30870/jppi.v5i2.5486 https://doi.org/10.1007/s40692-021-00210-y https://doi.org/10.1007/s10956-020-09870-4 Indonesian Review of Physics (IRiP) Vol.4, No.2, December 2021, pp. 61 - 69 68 Winarti, et al. Evaluation of STEM-Based Physics Learning on Students’ …. p-ISSN: 2621-3761 e-ISSN: 2621-2889 Rev. Psycho., vol. 73, no. 1, pp. 167-175, 2021. https://doi.org/10.1146/annurev-psych-020821-113042 [39] S. Prayogi and L. Yuanita, "Critical inquiry based learning: A model of learning to promote critical thinking among prospective teachers of physic," J. Turkish Sci. Edu., vol. 15, no. 1, pp. 43-56, 2018. https://doi.org/10.12973/tused.10220a [40] K. C. Yu, P. H. Wu, and S. C. Fan, "Structural relationships among high school students’ scientific knowledge, critical thinking, engineering design process, and design product," Int. J. Sci. Math. Edu., vol. 2, no. 1, pp. 1-22, 2019. https://doi.org/10.1007/s10763-019- 10007-2 [41] S. Sutoyo, U. Azizah, and S. Allamin, “Effectiveness of guided inquiry model integrated with STEM to improve the student critical thinking skills in chemistry learning,” Int. J. Innov. Sci. Res. Technol., vol. 4, no. 12, 2019. [42] N. Q. Linh, N. M. Duc, and C. Yuenyong, “Developing critical thinking of students through STEM educational orientation program in Vietnam,” J. Phys. Conf. Ser., vol. 1340, no. 1, 2019. https://doi.org/10.1088/1742- 6596/1340/1/012025 [43] N. R. Mater et al., “The effect of the integration of STEM on Critical thinking and technology acceptance model,” Educ. Stud., pp. 1–17, 2020. https://doi.org/10.1080/03055698.2020.1793736 [44] A. H. Prasadi, W. Wiyanto, and E. Suharini, “The implementation student worksheet based STEM and local wisdom to improve of critical thinking ability,” J. Prim. Educ., vol. 9, no. 3, pp. 227–237, 2020. https://doi.org/10.15294/JPE.V9I3.37712 [45] S. A. Annisa, A. D. Lesmono, and Y. Yushardi, “Comic- based module development andro-web to improve problem solving ability in physics high school students,” Berk. Ilm. Pendidik. Fis., vol. 8, no. 1, p. 40, 2020. http://dx.doi.org/10.20527/bipf.v8i1.7641 [46] A. Kholiq, “Development of B D F-AR 2 (physics digital book based augmented reality) to train students’ scientific literacy on global warming material,” Berk. Ilm. Pendidik. Fis., vol. 8, no. 1, p. 50, 2020. http://dx.doi.org/10.20527/bipf.v8i1.7881 [47] S. Mystakidis, A. Christopoulos, and N. Pellas, "A systematic mapping review of augmented reality applications to support STEM learning higher education," Edu. Inform. Tech., vol. 1, no. 1, pp. 1-45, 2021. https://doi.org/10.1007/s10639-021-10682-1 [48] D. F. Chang and H. C. ChangTzeng, "Patterns of gender parity in the humanities and STEM programs: The trajectory under the expanded higher education system," Stud. High. Edu., vol. 45, no. 6, pp. 1108-1120, 2020. https://doi.org/10.1080/03075079.2018.1550479 [49] A. Abdurrahman, N. Nurulsari, H. Maulina, and F. Ariyani, "Design and validation of inquiry-based STEM learning strategy as a powerful alternative solution to facilitate gift students facing 21st century challenging," J. Edu. Gift. Young Sci., vol. 7, no. 1, pp. 33-56, 2019. https://doi.org/10.17478/jegys.513308 [50] S. Ardianti, D. Sulisworo, Y. Pramudya, and W. Raharjo, "The impact of use STEM education approach on blended learning to improve student’s critical thinking skills," Univer. J. Edu. Res., vol. 8, no. 3B, pp. 24-32, 2020. https://doi.org/10.13189/ujer.2020.081503 [51] E. D. Heba, N. Mansour, M. Alzaghibi, and K. Alhammad, "Context of STEM integration in schools: Views from in-service science teachers," Eurasia J. Math. Sci. Tech. Edu., vol. 13, no. 6, pp. 2459-2484, 2017. https://doi.org/10.12973/eurasia.2017.01235a [52] M. B. Ibáñez and C. Delgado-Kloos, "Augmented reality for STEM learning: A systematic review," Comp. Edu., vol. 123, no. 1, pp. 109-123, 2018. https://doi.org/10.1016/j.compedu.2018.05.002 [53] D. Pertiwi, A. Suyatna, and E. Suyanto, “Enhancing critical thinking using LCDS-based interactive electronic school book in physics,” J. Pembelajaran Fis., vol. 7, no. 1, pp. 7–18, 2019. http://dx.doi.org/10.23960/jpf.v7.n1.201902 [54] S. MacDonald, "Introducing the model of cognitive- communication competence: A model to guide evidence- based communication interventions after brain injury," Brain Inj., vol. 31, no. 13, pp. 1760-1780, 2017. https://doi.org/10.1080/02699052.2017.1379613 [55] J. Afriana, A. Permanasari, and A. Fitriani, “Project based learning integrated to STEM to enhance elementary school’s students scientific literacy,” J. Pendidik. IPA Indones., vol. 5, no. 2, pp. 261–267, 2016. https://doi.org/10.15294/jpii.v5i2.5493 [56] A. Syawaludin, Gunarhadi, and P. Rintayati, “Development of augmented reality-based interactive multimedia to improve critical thinking skills in science learning,” Int. J. Instr., vol. 12, no. 4, pp. 331–344, 2019. https://doi.org/10.29333/iji.2019.12421a [57] S. S. Handoyo, T. Iriani, and E. Septiandini, “Study of the analysis on the characteristics of learning style of the students of the vocational education of building construction study program, faculty of engineering, Jakarta State University,” in 3rd Int. Conf. Tech. Voc. Edu. Train., 2019, vol. 3, no. 12, p. 339. https://doi.org/10.18502/kss.v3i12.4100 [58] S. Y. Foo and C. L. Quek, "Developing students' critical thinking through asynchronous online discussions: A literature review," Malaysian Online J. Edu. Tech., vol. 7, no. 2, pp. 37-58, 2019. https://doi.org/10.17220/mojet.2019.02.003 [59] S. Cargas, S. Williams, and M. Rosenberg, "An approach to teaching critical thinking across disciplines using performance tasks with a common rubric," Think. Skills Creat., vol. 26, no. 1, pp. 24-37, 2017. https://doi.org/10.1016/j.tsc.2017.05.005 [60] N. Maryanti, R. Rohana, and M. Kristiawan, "The principal’s strategy in preparing students ready to face the industrial revolution 4.0," Int. J. Edu. Rev., vol. 2, no. 1, pp. 54-69, 2020. https://doi.org/10.33369/ijer.v2i1.10628 [61] L. Varenina, E. Vecherinina, E. Shchedrina, I. Valiev, and A. Islamov, "Developing critical thinking skills in a digital educational environment," Think. Skills Creat., vol. 41, no. 1, pp. 100-108, 2021. https://doi.org/10.1016/j.tsc.2021.100906 [62] E. Cholisoh, “Upaya meningkatkan keterampilan berpikir kreatif dan keterampilan berpikir kritis ilmiah pada siswa dengan menggunakan model pembelajaran PJBL STEM pada materi termodinamika di kelas XI IPA 4 SMAN 10 Bandung semester ganjil tahun pelajaran 2018-2019,” in Pros. Sem. Nas. Fis. 5.0, 2019, pp. 59–73. [63] S. Han, R. Capraro, and M. M. Capraro, “How science, technology, engineering, and mathematics (STEM) project-based learning (PBL) affects high, middle, and low achievers differently: The impact of student factors on achievement,” Int. J. Sci. Math. Educ., vol. 13, no. 5, pp. 1089–1113, 2015. https://doi.org/10.1007/s10763- 014-9526-0 [64] A. K. Verma, D. Dickerson, and S. McKinne, “Engaging students STEM careers project-based learning-marinetech http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& https://doi.org/10.1146/annurev-psych-020821-113042 https://doi.org/10.1007/s10763-019-10007-2 https://doi.org/10.1007/s10763-019-10007-2 https://doi.org/10.1080/03055698.2020.1793736 https://doi.org/10.15294/jpe.v9i3.37712 http://dx.doi.org/10.20527/bipf.v8i1.7641 http://dx.doi.org/10.20527/bipf.v8i1.7881 https://doi.org/10.1007/s10639-021-10682-1 https://doi.org/10.1080/03075079.2018.1550479 https://doi.org/10.17478/jegys.513308 https://doi.org/10.12973/eurasia.2017.01235a https://doi.org/10.1016/j.compedu.2018.05.002 http://dx.doi.org/10.23960/jpf.v7.n1.201902 https://doi.org/10.1080/02699052.2017.1379613 https://doi.org/10.15294/jpii.v5i2.5493 http://dx.doi.org/10.18502/kss.v3i12.4100 https://doi.org/10.17220/mojet.2019.02.003 https://doi.org/10.1016/j.tsc.2017.05.005 https://doi.org/10.33369/ijer.v2i1.10628 https://doi.org/10.1016/j.tsc.2021.100906 https://doi.org/10.1007/s10763-014-9526-0 https://doi.org/10.1007/s10763-014-9526-0 Indonesian Review of Physics (IRiP) Vol.4, No.2, December 2021, pp. 61 - 69 69 Winarti, et al. Evaluation of STEM-Based Physics Learning on Students’ …. p-ISSN: 2621-3761 e-ISSN: 2621-2889 project,” Tech. Eng. Teach., vol. 71, no. 1, 2011. [65] D. A. budi Lestari, B. Astuti, and T. Darsono, “Implementasi LKS pendekatan STEM (science, technologi, engineering and matchmatic) untuk meningkatkan kemampuan berpikir kritis,” J. Pendidik. Fis. Dan Teknol., vol. 4, no. 2, pp. 202–207, 2018. http://dx.doi.org/10.29303/jpft.v4i2.809 [66] H. P. Rivai, L. Yuliati, and Parno, “Penguasaan konsep pembelajaran STEM berbasis masalah materi fluida dinamis pada siswa SMA [Mastery of STEM learning concepts based on dynamic fluid problems in high school students],” J. Pendidik. Teor. Penelitian, dan Pengemb., vol. 3, no. 8, pp. 1080–1088, 2018. http://dx.doi.org/10.17977/jptpp.v3i8.11481 [67] L. Wastiti and S. Sulur, “Pengaruh STEM- thinking maps pada model pembelajaran inkuiri terbimbing terhadap kemampuan berpikir kritis siswa kelas XI pada materi suhu dan kalor [The effect of STEM-thinking maps on the guided inquiry learning model on the critical thinking skills of class XI students on temperature and heat material],” J. Literasi Pendidik. Fis., vol. 4, no. 2, pp. 110–115, 2020. http://dx.doi.org/10.17977/um058v4i2p110-115 [68] M. Milner-Bolotin, H. Fisher, and A. MacDonald, “Modeling active engagement pedagogy through classroom response systems in a physics teacher education course,” Lumat Int. J. Math, Sci. Technol. Educ., vol. 1, no. 5, pp. 523–542, 2013. https://doi.org/10.31129/lumat.v1i5.1088 [69] O. Rogovaya, L. Larchenkova, and Y. Gavronskaya, “Critical thinking in STEM (science, technology, engineering, and mathematics),” Utop. y Prax. Latinoam., vol. 24, no. Extra 6, pp. 32–41, 2019. [70] R. Sagala, R. Umam, A. Thahir, A. Saregar, and I. Wardani, “The effectiveness of STEM-based gender differences: Impact of physics concept understanding,” Eur. J. Educ. Res., vol. 8, no. 3, pp. 753–761, 2019. https://doi.org/10.12973/eu-jer.8.3.753 [71] D. T. Tiruneh, M. De Cock, A. G. Weldeslassie, J. Elen, and R. Janssen, “Measuring critical thinking in physics: Development and validation of a critical thinking test in electricity and magnetism,” Int. J. Sci. Math. Educ., vol. 15, no. 4, pp. 663–682, 2017. https://doi.org/10.1007/s10763-016-9723-0 [72] A. Zollman, “Learning for STEM literacy: STEM literacy learning,” Sch. Sci. Math., vol. 112, no. 1, pp. 12–19, 2012. https://doi.org/10.1111/j.1949-8594.2012.00101.x http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526275227&1&& http://issn.pdii.lipi.go.id/issn.cgi?daftar&1526650381&1&& http://dx.doi.org/10.29303/jpft.v4i2.809 http://dx.doi.org/10.17977/jptpp.v3i8.11481 http://dx.doi.org/10.17977/um058v4i2p110-115 https://doi.org/10.12973/eu-jer.8.3.753 https://doi.org/10.1007/s10763-016-9723-0 http://dx.doi.org/10.1111/j.1949-8594.2012.00101.x