Microsoft Word - 1141 - Agnita Siska Pramasdyahsari (1-17) Available online at: https://journals.researchsynergypress.com/index.php/ijrse International Journal of Research in STEM Education (IJRSE) ISSN 2721-2904 (online) Volume 4 Number 2 (2022): 1-17 Corresponding author Agnita Siska Pramasdyahsari, agnitasiska@upgris.ac.id Faculty of Teacher Training and Education DOI: https://doi.org/10.31098/ijrse.v4i2.1141 Universitas Terbuka and Research Synergy Foundation Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati1, Agnita Siska Pramasdyahsari2, Iin Dwi Astutik3, Ukima Nusuki4, Sindi Nur Aini5, Julia Puspita Arum6, Wahyu Widodo7, Ummy Salamah8, Nafiatuz Zuliah9 1,2,3,4,5,6,7 Universitas PGRI Semarang, Indonesia 8 SEAMEO QITEP in Mathematics, Indonesia 9 SMP Barunawati Semarang, Indonesia Abstract Critical thinking is one of the important competencies in the twenty-first century that should be acquired. Therefore, there is a need to facilitate the students with the teaching approach and learning model that could improve this skill. One of the learning approaches that provide the students with the opportunity to think creatively is STEM-PjBL. Therefore, the purpose of the study is to describe the application of STEM-PjBL able to increase the critical thinking skills of the student. The SLR or Systematic Literature Review method was chosen to present a comprehensive and balanced summary of primary research results. The literature review was conducted through Google Scholar, Crossref, and Scopus. From 100 works of literature, 15 articles showed that PjBL STEM learning was able to improve critical thinking skills. This study examined 15 articles that were analyzed based on research methods, and the findings revealed that the application of STEM-PjBL could enhance the critical thinking skills of the student with the literature review method. Based on the existing data collection, it is analyzed through case studies that are the subject of discussion. The findings of the study revealed that: 1) the use of STEM-PjBL had a positive and significant impact on the critical thinking skills of students in each educational unit, including primary school, secondary schools, and high school; 2) the use of STEM-PjBL can help students improve their mathematical critical thinking skills; 3) interpretation, analysis, inference, evaluation, explanation, and self-regulation are among the indicators recommended for use as a reference for students' mathematical critical thinking skills. Keywords: Critical Thinking Skills; Improving; STEM-PjBL This is an open access article under the CC–BY-NC license. INTRODUCTION The learning process is expected to meet 21st-century competencies, namely 4C consisting of creative thinking skills, critical thinking skills, collaboration, and communication. These competencies are integrated into the independent curriculum which has been implemented by the schools. Risdianto (2019) said that the implementation of the independent learning curriculum aims to follow education’s challenges in the era of the industrial revolution 4.0. The realization of this curriculum supports the skills that students should have, such as critical thinking in solving problems, being creative and innovative, and being skilled in communicating and collaborating. Critical thinking is the basic skill that individuals must possess in the learning proses (Özyurt, 2015). International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah 2 │ ISSN 2721-2904 (online) However, students critical thinking ability is still low (Nuryanti, 2018). At the same time, the students have the potential to master concepts and thinking skills. As stated by Yustyan et al. (2016), basically, students have skills in mastering concepts, good thinking skills, as well as critical thinking skills in learning, but sometimes these skills are not can develop well. The interpretation, analysis, evaluation, inference, explanation, and self-regulation serve as guidelines for critical thinking ability tests to be analyzed (Fithriyah et al., 2016). The findings revealed that students' critical thinking abilities remain low. This is because students are not able to understand the problem well. These findings indicate that these studies can provide information to teachers about the critical thinking skills of students so that teachers are expected to improve the critical thinking skills of each student by designing the learning activities. Susilowati et al. (2017) also stated that the student's critical thinking skill profile is low; therefore, it is expected that teachers will be able to design learning activities that empower students’ critical thinking skills. The selection of appropriate learning strategies can improve the critical thinking skills of students. One of the choice strategies that can be implemented is STEM (Science, Technology, Engineering, and Mathematics)- Project-based Learning (PJBL) which is integrated into the learning process. STEM is a multidisciplinary learning approach that all students can apply in real-life contexts that connect school, their own work, and the world. As a consequence, using STEM literacy helps prepare all students to participate in a new and original knowledge-based economy era. Furthermore, STEM allows students to engage as objects in higher cognitive activities. STEM has an implementation in the classroom that enables the students to comprehend the significance of integrating different disciplines along with their own applications (Anwari et al. 2015, in Murnawianto, et al. 2017). Project Based Learning is a knowledge approach that strives to be project-based and student- centered during in the learning process. The PjBL model allows students to plan their own many learning activities, collaborate on projects, and generate quality work products that can be presented to others (Kemendikbud, 2017). According to Suranti et al. (2016), PjBL is one of the motivation factors for students to take part in investigations, collaborate on research, and create projects that apply their own knowledge to continue to innovate, master the application of technology, and solve difficulties in the form of innovative learning. Considering the important role of STEM and PJBL learning models in stimulating students' critical thinking; therefore, there is a need to know how the implementation of STEM-PjBL could improve the students’ mathematical critical thinking. LITERATURE REVIEW What exactly is STEM? STEM refers to Science, Technology, Engineering, and Mathematics (Reeve, 2013 in Yanuar, 2018; Devi et al., 2018; and Toto, 2019). STEM is a multidisciplinary research learning approach in which the students utilize STEM in a meaningful context that completely integrates at school, work, and the world, allowing students to compete in a unique knowledge economy era. Students' logical thinking skills can benefit from the STEM approach. According to Anwari et al. (2015, in Murnawianto et al., 2017), STEM allows students to engage in metacognitive tasks. This metacognitive activity encompasses the steps of selecting, searching, asking, sharing, hypothesizing, and making decisions (Sularmi, et al., 2018). International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah │ 3 ISSN 2721-2904 (online) STEM provides opportunities for learners to comprehend the significance of integrating various areas of study and their personal applications in learning through its implementation. As a consequence of the implementation, students can enhance their own logical thinking. In line with it though, STEM can assist students in increasing their capacity to address any problem which is presented and overcome it using critical thinking. According to a different viewpoint expressed by Murnawianto et al. (2017), STEM education seems to have a comprehensive character (critical analysis and problem-solving) in providing opportunities for many students to demonstrate their possess thinking skills. Through the characteristics of its approach, STEM education has a wonderful chance to develop students' thinking skills. The STEM framework does have objectives that correspond with the characteristics of 21st- century education, among which are critical thinking skills, or thinking that is always curious about the information available in order to achieve a thorough understanding. As a result, learning using the STEM approach can assist students in enhancing their ability to think critically. According to Yanuar (2018), 21st-century competencies that can be established through STEM learning encompass critical thinking, creative thinking, communication, and collaboration. Using the PjBL or Project Based Learning model in combination with the STEM approach (the term used by researchers is STEM-based) should improve students' critical thinking skills. This could occur as a result of activities pressured out in learning which are more oriented to students ’ participation, which could also encourage students to reflect critically. Thus, using the PjBL STEM (Science, Technology, Engineering, and Mathematics) model can assist students in enhancing their critical thinking skills. It motivates students while also demanding them, according to Capraro et al. (2013, in Afriana et al., 2016), because it trains them to analyze, think critically, and strengthening higher- level thinking skills. Thinking abilities are necessary for dealing with life's challenges. That includes critical thinking abilities, creative thinking skills, and skills for problem-solving (Kalelioglu & Gulbahar, 2014). Critical thinking is one of the skills required to deal with personal and social problems. Here are some examples of critical thinking definitions. According to Facione (2011), critical thinking is the ability to organize and generate evaluations, interpretations, inferences, and analyses, as well as reveal concepts, criteria, evidence, methodologies, and or contextual considerations as the basis for making decisions. According to Choy and Cheah (2009), critical thinking is required to process cognitive information at a high level and to define complex processes. Critical thinking, according to Ennis (2011), is one of the skills of reflective reasoning and thinking that is concentrated on what has been believed or completed. Critical thinking skills involve original clarification, basic decision- making process, inference, providing an extra explanation, estimation, and incorporation, in addition to other abilities. Critical thinkers could indeed evaluate and analyze such data that is presented to them. According to Duron et al. (2006), becoming able to evaluate data and analyze, inform crucial questions and problems, formulate hypotheses and problems until they are clear, and collect and evaluate relevant information. Moreover, a critical thinker can use abstract ideas to find relevant information, be open-minded, and effectively communicate with others. International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah 4 │ ISSN 2721-2904 (online) RESEARCH METHOD In this study, a Systematic Literature Review (SLR) is used, which is a thorough and balanced mixed research method that is encapsulated from the primary research results. The Systematic Literature Review method could indeed recognize journal articles in a structured manner, for each process following the sequence that has been established (Thovawira et al.,2021). SLR aims to locate and synthesize studies comprehensively. This also refers to particular questions, utilizing procedures that are structured, straightforward, and repeatable throughout every stage of the process (Juandi, 2021). SLR was employed in STEM to recognise, critically assess, and synthesize research results from across all relevant research, in addition, to describing learning and instruction in integrated STEM (Thibaut et al., 2018). Developing research questions (formulating research questions), developing the search strategy (looking for articles or literature which match the research topic), selection process (applying integration method to identify articles), evaluating and analyzing data, and interpreting (reporting research findings) are all steps in the SLR (Andani et al., 2021). Data were gathered from articles in the databases Semantic Scholar, Google Scholar, Education ERIC or Resources Information Center, and Directory Open Access Journal. The articles chosen are those that are relevant to the research question. The keywords used in the search for articles are "STEM PjBL to improve critical thinking skills”. The inclusion criteria for this study were figured out in figure 1 as follows: articles related to the theme, that is, STEM in Mathematics learning, articles variety of different settings used in the STEM approach, or articles containing the integration of STEM into learning models, articles published between 2018 and 2022, articles from national journals and international indexed or proceedings, and articles from national journals and international indexed or proceedings. The following step is to choose and evaluate articles. The selection of articles that meet the inclusion criteria is pushed out at this stage. Only articles that are both relevant and fulfill the inclusion criteria will be analyzed (Juandi, 2021). Articles that do not meet the inclusion requirements are not taken into account for the following stage. Articles and journals that meet the inclusion criteria are then coded and sorted for subsequent review based on their relevance to the theme. The final step is to present the research findings. In this step, the research results are summarized in a systematic and clear manner. International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah │ 5 ISSN 2721-2904 (online) The article search results are depicted in the chart below Figure 1. The Protocol of Systematic Review FINDINGS AND DISCUSSION Based on the protocol of systematic literature review, the data was gathered as described in Table 1. Table 1 elaborates on the calculation of STEM PjBL in critical thinking skill effect size. Table 1. The Calculation of STEM PJBL in Critical Thinking Skills Effect Size No Author Journal/ Proceeding Research Subject Research Method Indicator Critical Thinking Skill Result 1 Afifah, Afni Nur Ilmiyati, Nur Toto, Toto (Afifah et al., 2019) Quagga: Jurnal Pendidikan dan Biologi Google Scholar pre - eksperiment al method There are several aspects of inference, analysis, explanation, self- regulation, and evaluation of critical thinking indicators. The study shows that PJBL STEM could improve the student's grasping of the concept and improve critical thinking. 2 Eja Ramalis, T. R. Suwarma, I. R. (Eja et al., 2020) Journal of Physics: Conference Series Scopus 34 students This study involved the quantitative descriptive research method. This study uses critical thinking skill indicators consisting analysis, inference, explanation, To process information in terms of cognition at a high level and to define a process at a complex Research Question Database searching using the keyword stem-pjbl in critical thinking (2018-2022) Database: Scopus (10), Crossref (10), and Google Scholar (80) Relevance by abstract Scopus (4), Crossref (6), and Google Scholar (5) Total : 15 Inclusion and exclusion criteria Scopus (4), Crossref (6), Google Scholar (5) Total : 15 Synthesis and Results Assessment of quality, data extraction) Total : 15 Relevance by Title Scopus (4), Crossref (6) and Google Scholar (5) Total: 15 International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah 6 │ ISSN 2721-2904 (online) No Author Journal/ Proceeding Research Subject Research Method Indicator Critical Thinking Skill Result evaluation, and interpretation level requires critical thinking 3 Parno Nur'aini, D. A. Kusairi, S. Ali, M.(Parno, 2022) Journal of Physics: Conference Series Scopus class XI in Malang, Indonesia quasy experimenm ethode This essay test is in accordance with what is in critical thinking skills This study recommends further research to put an ART aspect for STEM to improve the critical thinking skill of the student. 4 Bulu, Vera Rosalina Tanggur, Femberianus( Bulu & Tanggur, 2021) Al-Jabar : Jurnal Pendidikan Matematika Crossreff This study was conducte d in the first semester of the 2020/202 1 academic year in PGSD (Primary School Teacher Education Program) of Citra Bangsa Universit y, Kupang. This Descriptive method and quantitative research were used in this study. In learning mathematics, students should have critical thinking skills because it can help them to analyze a problem, provide critical responses, and find solutions. PjBl with STEM Effectively improves critical and collaborative thinking ability 5 Rosyidah, Nur Diana Kusairi, Sentot Taufiq, Ahmad(Rosyi dah et al., 2021) Jurnal Pendidikan: Teori, Penelitian, dan Pengembang an Google Scholar siswa kelas XI MIPA 6 SMAN 1 Batu (N=32 siswa). Research design used mixed methods As the previous study, this research also employ indicator such as Analysis, interpretation, explanation, Evaluation and inference This study shows that students' critical thinking ability improves at all of the indicators. 6 Rahmawati, Yuli AIP Conference Proceedings year 10 students a multimed A qualitative methodology Criteria for critical thinking include the rights The analysis of the critical thinking ability International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah │ 7 ISSN 2721-2904 (online) No Author Journal/ Proceeding Research Subject Research Method Indicator Critical Thinking Skill Result Hadinugrahan ingsih, Tritiyatma Ridwan, Achmad Palimbunga, Uyun Sugiarti Mardiah, Alin(Rahmaw ati et al., 2021) Scopus ia major at a vocationa l high school in Banten Province. used to identify the question in question, students' conceptual understanding, and their ability to relate and create ideas, assumptions, and conclusions. assessment guide shows that students can identify the questions asked and make conclusions at a good level, while at a sufficient level at the conceptual- based understanding step, the connection of basic ideas. 7 Rahmania, Ika(Rahmania , 2021) Budapest International Research and Critics Institute (BIRCI- Journal): Humanities and Social Sciences crossref - Literatur review Critical skills emerged in this study which was measured consisting of five things, namely interpretation, analysis, evaluation, explanation, and inference Pjbl and STEM can bring up the ability to think critically and creatively, systematically and logically 8 Khotimah, Rita Pramujiyanti Adnan, Mazlini Ahmad, Che Nidzam Che Murtiyasa, Budi(Khotima h et al., 2021) Journal of Physics: Conference Series Scopus - Literatur review Five aspects are used to determine the critical skill item; there are interpretation, analysis, evaluation, explanation, and inference STEM education in Indonesia is one of the most effective in student learning outcomes. 9 Isro, Abda Lail Anggraito, Yustinus Ulung Bintari, Siti Harnina (Isro et al., 2021) Journal of Innovative Science Education Google Scholar MA Tahfizhul Qur'an As Surkati Salatiga a mix- method to demonstrate critical thinking skills such as basic clarification inference, basic support, advanced clarification, and This study demonstrates that students in PjBL STEM already have high critical thinking skills. International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah 8 │ ISSN 2721-2904 (online) No Author Journal/ Proceeding Research Subject Research Method Indicator Critical Thinking Skill Result strategic and tactical aspects. An essay test is used to collect information. 10 Setyowati*, Yeni Kaniawati, Ida Sriyati, Siti Nurlaelah, Elah Hernani, Hernani (Setyowati et al., 2020) Jurnal IPA & Pembelajara n IPA crossref of junior high school students in class VII. Research and development (R&D) to develop a design model, define, distribute, and develop a Four-D development model This study shows that the teaching materials developed were valid and suitable for use in learning. 11 Dywan, Almahida Aureola Airlanda, Gamaliel Septian (2020) Jurnal Basicedu Crossref SD Negeri Dukuh 01 Salatiga dan SD Negeri Kecandra n 01 Salatiga Quasi Eksperiment al method STEM-PJBL is used because it is felt that it can improve critical thinking skills in terms of effectiveness 12 Sulisworo, Dwi Kaliappen, Narentheren History, Article Winarti, W Sulisworo, D Kaliappen, N (Sulisworo et al., 2021) Indones. Rev. Phys Crossref - Literatur review This study demonstrates that students in PjBL STEM already have high critical thinking skills. This study aims to improve students' critical thinking skills. 13 Fithriyah, Inayatul Sa’dijah, Cholis Sisworo (Fithriyah et al., 2016) Prosiding Konferensi Nasional Penelitian Matematika dan Pembelajara nnya Google Scholar 26 siswa kelas IX-D SMPN 17 Malang pada semester ganjil tahun pelajaran 2014/201 5 a qualitative descriptive The student lacked critical thinking skills. Students' critical thinking abilities still need to be developed. International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah │ 9 ISSN 2721-2904 (online) No Author Journal/ Proceeding Research Subject Research Method Indicator Critical Thinking Skill Result 14 Kurniasih, Rini Hakim, Dori Lukman (Kurniasih et al., 2019) Prosiding Seminar Nasional Matematika dan Pendidikan Matematika Google Scholar - literature review containing relevant scientific studies to the problem studied. Critical thinking skills help students interpret, analyze, discover, and conclude. Students in the ability to think critically on quadrilateral material. 15 Edi, Sarwo Rosnawati, Raden (2021) JNPM (Jurnal Nasional Pendidikan Matematika) Crossref peserta didik kelas XI MIPA 2 di SMA Negeri 1 Tempel Model ADDIE The activities of interpretation, inference, analysis, and evaluation demonstrate students' critical thinking skills. Critical thinking skills that are integrated into lesson plans and LKPD are used as learning tools. Afifah et al. (2019) discovered the STEM PjBL model for program enhancement in terms of concept mastery and critical thinking skills through students. This study's population consisted of students in grade 10 from Madrasah Aliyah Negeri in Ciamis Regency, split into four classes. The study employed the purposive sampling method on one class. The pre-experimental method was used in this study. A multiple choice test and a critical thinking skill test in the form of critical thinking skills test questions adapted for concept mastery in students are used as instruments. The research data were statistically analyzed. Based on the findings of the study, it is possible to enhance conceptual knowledge and understanding (medium) and critical thinking skills (high) by using the STEM-based PJBL model. Fitriyah and Ramadani (2021) found that PjBL-based STEAM learning has a significant effect on students' creative thinking skills with a sig. of 0.000 less than 0.05 and a Fcount of 35.551 and that PjBL-based STEAM learning has a significant effect on student's critical thinking skills with a sig. of 0.003 less than 0.05 and a Fcount of 9.401. This is due to the fact that combining PjBL and STEAM can be a learning innovation that generates creative and critical ideas and solutions, making it easier to solve a problem. As a result, educators are strongly advised to use the STEAM PjBL learning model as an innovative learning model in schools. Dywan and Airlanda (2020) investigate the impact of PJBL models or STEM-based projects (Science, Technology, Engineering, Mathematics) on critical thinking skills. The population of the Gajah Mada cluster, Sidomukti District, Salatiga city, and use a sample of 32 students from SD Negeri Dukuh 01 Salatiga as the experimental group and 33 students from SD Negeri Kecandran 01 Salatiga as the control group. This research employed a semi-experimental research design with a non- equivalent control group design to determine which experimental studies should be performed before and after the others. The results of the study were collected using the independent t-test in SPSS 20 for Windows with a significance level of 0.038 & lt; 0.05, indicating that there is a significant International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah 10 │ ISSN 2721-2904 (online) effect in which H0 is rejected, and H1 is accepted and that the average pre-test for the post-test in the experimental group increases by 13.38 while it increases by 7.51 in the control group. The STEM PJBL-based learning model is the most effective in improving the critical thinking skills of 4th-grade elementary school students about science learning content. According to Eja et al. (2020), 25% of students' critical thinking abilities were in the low category, 34% of students' critical thinking abilities were in the medium category, and 41% of students' critical thinking abilities were in the high category. Peer assessment backs up this result, which shows 12% in the low category, 41% in the medium category, and 47% in the high category. As a result, the study's conclusion is that students' critical thinking processes during PjBL-STEM learning produce excellent results. Parno (2022) reported that his effect size is 1.13 (Very Large Typeshows), indicating the need for AF STEM PjBL learning to be carried out more frequently in the field. Students in the comparison group had no more positive learning responses than students in the experimental group. In future studies, "Art" is highly recommended to be incorporated into the STEM approach in order to further develop students' CTS. Bulu and Tanggur (2021) The purpose of this study is to determine how the PjBL STEM model affects students' cooperative attitudes and critical thinking skills. This study is a quantitative study using a descriptive method. Data was gathered using critical thinking skills tests, collaborative attitude questionnaires, and observational learning guides. The researcher must perform a preliminary test consisting of the normality test, the homogeneity test, the covariance-matrix homogeneity test, and the N-test before analyzing the data. Gain and efficiency testing The results of the basic hypothesis test indicate that the MANOVA test can be used. MANOVA results indicate that PjBL STEM is an effective learning model for improving students' critical thinking skills and cooperative attitudes. According to Rosyidah et al. (2021), the integrated experimental design was used in mixed methods research, and the sample size was 32 students. According to the findings of the analysis, the activities carried out can help students improve their critical thinking skills. Overall, students improved in all critical thinking indicators. The STEM approach is adopted by incorporating a project-based learning model. When students reach a sufficient level of conceptual understanding, connecting assumptions and ideas as demonstrated by the critical thinking skills rubric analysis, they are able to identify problems and draw conclusions at a high level. Time management, a lack of student participation, and project development that must be relevant to the subject of study are all issues. According to Rahmania (2021), the findings of this research demonstrate that a number of the initial skills in junior secondary school could be integrated into a STEM approach to learning to provide projects for students. As a result, STEM can be utilized in scientific learning in junior high school students to prepare them for the twenty-first century's signature. Human resources, which could really think critically and creatively, systemically and logistically, can be oriented in PJBL and STEM approaches to satisfy human standards in the twenty-first century and investigate challenging issues. Khotimah et al. (2021) findings indicate that: 1) Project-Based Learning (PjBL) within STEM in Indonesia as a Carry outer, 6E, Assessment-Based Learning HOTS, Inquiry, Paired Thinking (TPS), Problem-Based Learning (PBL), PjBL is the most widely used learning model in the implementation of STEM education in Indonesia, with Android games, digital learning, and student International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah │ 11 ISSN 2721-2904 (online) books. 2) STEM education is implemented in elementary, middle, and high schools in Indonesia but is very limited at the university level. 3) In Indonesia, STEM education has a significant impact on student learning outcomes such as scientific knowledge, creative thinking, critical thinking, attitude, HOTS, personality, achievement, problem-solving, and 21st-century skills. According to research, 1) work STEM-PjBL implementation has a significant and positive impact on student's critical thinking skills at all educational levels, including elementary, middle, and high school. 2) STEM-PJBL can help students improve their mathematical critical thinking skills. 3) The following indicators are recommended as benchmarks for assessing students' critical thinking skills in math: interpret, analyze, draw a conclusion, evaluate, interpret, and self-correct. Isro and colleagues (2021) revealed that students could demonstrate diversity in their critical thinking skills. All aspects of all strategies and tactics fall into this category. The top category includes explanations of basic aspects, basic support, and conclusions. The first step is to define the type of intermediary. The critical thinking of students in relation to environmental change learning materials PjBL STEM has a relatively high average. In Setyowati et al. (2020) study, the topic of STEM PjBL became a topic for instructional content as a method of increasing students' critical thinking and awareness in a lengthy period and sustainability. Based on the feasibility test results, a score of 85 with a very strong grounding for the critical thinking ability test and a score of 97 with a strong background for the sustainability perception test. The PjBL-STEM model and the ESD approach to pollution serve as the foundation. The creation of learning aids is an effort to enable teachers to use ESD as a scientific learning method in high school. Dywan, Aureola, Airlanda, and Septian (2020) examine the effects of PJBL-based (project- based learning), STEM (science, technology, engineering, and math), and non-STEM-based learning models on skills. Fourth-year elementary school students' critical thinking. The population of the Gajah Mada cluster, Sidomukti district, Salatiga city, used SD Negeri Dukuh 01 Salatiga sample as an experimental group of 32 students and SD Negeri Kecandran 01 Salatiga sample as a control group of 33 students. To ascertain test accomplishment, this study used a semi-experimental study design with a non-equivalent control group and required pre- and post-trial trials. An independent t-test with a significance level of 0.038 produced the results of this study. 0.05 when using SPSS 20 for Windows. This means that H1 is accepted and H0 is rejected, which has a significant effect on increasing the mean pre-test to post-test scores in the experimental group from 7.51 to 13.38. To enhance 4th-grade elementary school students' critical thinking skills by using the most effective science-related content and the PjBL STEM-based learning model. The findings show that STEM is taught at the high school and undergraduate levels in Indonesia, according to Sulisworo et al. (2021). In addition, relevant research on STEM-based physics with learning media. This study has several limitations, including the fact that the main reference material used as the main research material is still limited to Indonesian research. The findings of the Indonesian language study cited in this article are not from peer-reviewed international journals. Other researchers should use critical thinking skills to demonstrate the performance of various learning materials, learning models, and scientific bibliographies published in reputable international journals to ensure universal use. International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah 12 │ ISSN 2721-2904 (online) According to Fithriyah et al. (2016), SMP VIII students' critical thinking ability remains low. The low average score of students in the correct category demonstrates this (B). Students' low critical thinking ability is caused by a lack of exposure to active learning, which maximizes students' potential. The findings of this study provide teachers and researchers with an overview of the critical thinking skills of young college students. Teachers must be more creative in designing and developing learning tools to help students become more familiar with critical thinking. Through various active learning models, teachers should engage students in learning situations that stimulate their critical thinking skills. Kurniasih and colleagues (2019) describe that critical thinking teaches us when solving the tetrahedron problem various descriptions can be found. Critical thinking abilities are required to educate students in math and problem-solving abilities. However, the students remain unable to answer the questions. When the researcher asked the question, the students were still undecided about responding to the 4-page document question. Average students lack analytical ability because they do not comprehend the fundamentals of the problem, therefore, are unable to create mathematical models. As a result, they continue to be incorrect when providing explanations or solving problems. Computational steps are taken in the skills assessment for incorrect answers because students are still confused about how to implement the strategies used. Furthermore, students remain incapable of drawing correct conclusions from questions about their ability to formulate conclusions because they are unable to make judgments based on concepts and concepts. Based on Edi, Rosnawati, and Raden (2021) we can learn a lot from the description of critical thinking when solving the tetrahedral problem, and various descriptions emerge. Critical thinking skills are required to train students in the solving and solving of mathematical problems. Students continue to be unable to interpret the questions. When the researcher asked, students were still perplexed about completing the 4-page material question. On average, students with Analytical Competence do not understand the concepts in the problem, so they are unable to create mathematical models. As a result, they continue to provide incorrect explanations or solve problems. Because students are still confused in implementing the strategies used, computational steps are performed in the competency assessment for incorrect answers. Furthermore, students cannot draw correct conclusions from the questions asked in relation to their ability to draw conclusions; this is because students still cannot draw conclusions based on concepts and concepts. The description of critical thinking teaches us a lot. Only 15 STEM-PjBL compliance exercises can improve critical thinking skills, including 5 research papers with quantitative methods, 2 articles with qualitative research methods, 2 articles with mixed methods, 2 research papers with research and development models, and 4 literature reviews. Each article suggests that STEM-PjBL can assist students to improve their critical thinking skills. The aspects of critical skill measured in this research included five aspects: interpretation, analysis, evaluation, interpretation, and inference (Facione, 2007). The Trend of Improving critical thinking in STEM-PjBL Research in Last 7 Year-Publication Figure 2 depicts search trends over the last seven years based on VOSviewer-processed bibliographic data. Colors that are the same color represent the same group, and the size of the circle represents keyword popularity. The larger the circle, the more frequently the topic is covered International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah │ 13 ISSN 2721-2904 (online) in the 34 articles. The figure appears to be able to connect the lines between the circles as well as the keywords directly. Figure 2. The Keyword Network Visualization of the 34 Processed Articles Categories of keywords or variables that occur frequently and their relationships Being able to observe the most frequently studied topics over a given period of time could unveil research trends on critical thinking skills in STEM-PjBL learning over the last 7 years. Comparisons were also made to research trends in critical thinking in STEM-PjBL learning in order to identify developmental milestones. This can be seen in topics that have only been around for a short time, as well as the most popular ones. There are four keyword groups that reveal related keywords in the processed articles. Current keywords include critical thinking skills in STEM-PjBL learning, critical thinking skill indicators, and research participants. The popularity of the 34 articles mentioned is indicated by each circle-sized keyword. The larger the size and the circle indicate more keyword usage in search. This implies that the previous variable was searched. Figure 3 illustrates the direct relationship between the keyword "critical thinking skills in STEM- PjBL learning" and other keywords. Figure 3. The Relation of the 'critical thinking skill in STEM-PjBL learning’ keyword with other keywords International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah 14 │ ISSN 2721-2904 (online) Figure 3 illustrates the inclusion of the keywords "critical thinking and STEM-PjBL" in group 2 with 10 links and 4 facts. The 21st Century Skills, Student Critical Thinking, PjbL-STEM, and Project Model, are keywords related to "Critical Thinking Skills and STEM-PjBL." These keywords are directly related to others in three groups, namely those in group 2, namely 6 (blue) and 5 (orange). This means that, in 34 studies, keywords from the four groups were more likely to appear as a topic in the search title. Figure 4. Publication Trend from 2015 to 2022 Color demonstrates trends, as shown in Figure 3, with lighter marks indicating the most recent posts. The most recent publications, highlighted in yellow, show that keywords such as integrated project, STEM integrated PjBL, and STEM model PjBl is trending. For researchers, information about the topic's novelty is critical in representing the current state of studies carried out throughout time. Benchmark for students’ mathematical critical thinking skill Regarding the previous research employing the STEM PjBL to know its impact on the critical thinking skill (Afifah et al., 2019; Rahmania, 2021; Edi & Rosnawati, 2021; Khotimah et al., 2021); thus, there are five aspects used to determine the critical skill item. There are interpretation, analysis, evaluation, explanation, and inference were first introduced by Facion (2011) as indicators of critical thinking skills. Each indicator, called the metrics, is described in table 2. Table 2. Metrics indicators critical thinking skill Indicators Description Interpretation Understanding how to convey significance or meaning based on a variety of personal experiences, circumstances, information, occurrences, evaluations, behavioral patterns, traditions, opinions, regulations, methods, or requirements. International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah │ 15 ISSN 2721-2904 (online) Indicators Description Analysis Actual referential correlation is intended for questions, declarations, notions, explanation, or other depiction that aims to convey opinions, decisions, experiences, reasons, data, or viewpoints. Evaluation Reviewing credibility of questions or other representation in the form of reports or descriptions from perceptions, experiences, situations, judgments, beliefs, or opinions, and interpreting logical power of referential correlation or other intended representation. Explanation Examining the validity of questions or other types of representation derived from perceptions, experiences, circumstances, decisions, belief systems, or thoughts and analyzing the rational authority of referential correlation or other intentional representation. Inference Recognize and obtain the components required to reach a logical conclusion, such as making assertions and assumptions, contemplating relevant data, and drawing conclusions from data, circumstances, questions, and other representations. CONCLUSION Based on the findings and discussion of this literature review, several conclusions have been drawn: 15 articles from 100 documents show that learning PjBL STEM can improve critical thinking skills. This is a literature review in which 15 articles are analyzed using research methods, and the results show that the use of STEM-PjBL can enhance students' critical thinking skills. The articles in the dataset were analyzed using a case study. According to the findings of the study, (1) the use of STEM-PjBL has had a positive and significant impact on the critical thinking skills of students in each educational unit, including elementary, middle, and high schools; (2) The use of STEM-PjBL can assist students in enhancing their mathematical critical thinking skills.; and (3) The following metrics are recommended as benchmarks for students' mathematical critical thinking skills: interpretation, analysis, inference, evaluation, interpretation, and self-correction. LIMITATION & FURTHER RESEARCH The findings of this systematic literature review have some limitations. To begin with, the literature is only searched from 2018 to 2022. Only STEM- PjBL can improve critical thinking skills are the two keywords. Therefore, there is a need to add several keywords from aspects of psychological mathematics related to STEM-PjBL and the skills for twenty-first learning, such as mathematical literacy, collaborative, computational thinking, problem-solving, and creative thinking. REFERENCES Afifah, A. N., Ilmiyati, N., & Toto, T. (2019). Model Project Based Learning (PJBL) Berbasis Stem Untuk Meningkatkan Penguasaan Konsep Dan Keterampilan Berpikir Kritis Siswa. Quagga: Jurnal Pendidikan Dan Biologi, 11(2), 73. https://doi.org/10.25134/quagga.v11i2.1910 Andani, M., Haki Pranata, O., & Hamdu, G. (2021). Pedadidaktika: Jurnal Ilmiah Pendidikan Guru Sekolah Dasar Systematic Literature Review: Model Problem Based Learning pada Pembelajaran Matematika Sekolah Dasar. All Rights Reserved, 8(2), 404–417. http://ejournal.upi.edu/index.php/pedadidaktika/index International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah 16 │ ISSN 2721-2904 (online) Bulu, V. R., & Tanggur, F. (2021). The Effectiveness of STEM-Based PjBL on Student’s Critical Thinking Skills and Collaborative Attitude. Al-Jabar : Jurnal Pendidikan Matematika, 12(1), 219–228. https://doi.org/10.24042/ajpm.v12i1.8831 Choy, S. C., & Cheah, P. K. (2009). Teacher Perception of Critical Thinking Among Students and Its Influence on Higher Education. International Journal of Teaching and Learning in Higher Education, 20(2), 198—206. Retrieved from http://www.isetl.org/ijtlhe/pdf/IJTLHE336.pdf. Duron, R., Limbach, B., & Waugh., W. (2006). Critical Thinking Framework for Any Discipline. International Journal of Teaching and Learning in Higher Education, 17(2), 160—166. Retrieved from http://www.isetl.org/ijtlhe/pdf/IJTLHE55.pdf Edi, S., & Rosnawati, R. (2021). Kemampuan Berpikir Kritis Siswa dalam Pembelajaran Matematika Model Discovery Learning. JNPM (Jurnal Nasional Pendidikan Matematika), 5(2), 234. https://doi.org/10.33603/jnpm.v5i2.3604 Eja, Ramalis, T. R., & Suwarma, I. R. (2020). Profile of students’ critical thinking ability in project based learning integrated science technology engineering and mathematics. Journal of Physics: Conference Series, 1521(2). https://doi.org/10.1088/1742-6596/1521/2/022042 Ennis, R. H. (2011). The Nature of Critical Thinking: An Outline of Critical Thinking Disposition and Abilities. Last Revised. Emeritus Professor: University of Illinois. Facione, P. A. (2011). Critical Thinking: What It Is and Why It Counts. Millbrae: Measured Reasons and The California Academic Press. Fithriyah, I., Sa’dijah, C., & Sisworo. (2016). Analisis Kemampuan Berpikir Kritis. Prosiding Konferensi Nasional Penelitian Matematika Dan Pembelajarannya, 2006, 155–158. Isro, A. L., Anggraito, Y. U., Bintari, S. H., & Semarang, U. N. (2021). Description of student's critical thinking skills in integrated PJBL STEM learning environmental change material. Journal of Innovative Science Education, 10(37), 237–243. https://journal.unnes.ac.id/sju/index.php/jise/article/download/43920/17871 Juandi, D. (2021). Heterogeneity of problem-based learning outcomes for improving mathematical competence: A systematic literature review. Journal of Physics: Conference Series, 1722(1). https://doi.org/10.1088/1742-6596/1722/1/012108 Kurniasih, R., & Hakim, D. L. (2019). Berpikir kritis siswa dalam materi segiempat. Prosiding Seminar Nasional Matematika Dan Pendidikan Matematika, 2017, 1135–1145. https://journal.unsika.ac.id/index.php/sesiomadika/article/view/2911, diakses Kamis 4 Maret 2021 pukul 16:52:02 WIB Kaleiloglu, F., & Gulbahar, Y. (2014). The Effect of Instructional Techniques on Critical Thinking Disposition in Online Discussion. Educational Technology & Society, 17(1), 248—258. Kemendikbud. (2017). Buku Guru Ilmu Pengetahuan Alam. Jakarta: PT.Tiga Serangkai Pustaka Mandiri. Khotimah, R. P., Adnan, M., Ahmad, C. N. C., & Murtiyasa, B. (2021). Science, Mathematics, Engineering, and Mathematics (STEM) Education in Indonesia: A Literature Review. Journal of Physics: Conference Series, 1776(1). https://doi.org/10.1088/1742-6596/1776/1/012028 Özyurt, Ö. (2015). Examining the critical thinking dispositions and the problem solving skills of computer engineering students. Eurasia Journal of Mathematics, Science and Technology Education, 11(2), 353–361. https://doi.org/10.12973/eurasia.2015.1342a International Journal of Research in STEM Education (IJRSE), Vol. 4 (2), 1-17 Improving Mathematical Critical Thinking Skill through STEM-PjBL: A Systematic Literature Review Rina Dwi Setyawati, Agnita Siska Pramasdyahsari, Iin Dwi Astutik, Ukima Nusuki, Sindi Nur Aini, Julia Puspita Arum, Wahyu Widodo, Ummy Salamah, Nafiatuz Zuliah │ 17 ISSN 2721-2904 (online) Parno. (2022). Impact of The STEM approach with formative assessment in PjBL on students’ critical thinking skills. Journal of Physics: Conference Series, 2165(1). https://doi.org/10.1088/1742-6596/2165/1/012044 Rahmania, I. (2021). Project Based Learning (PjBL) Learning Model with STEM Approach in Natural Science Learning for the 21st Century. Budapest International Research and Critics Institute (BIRCI-Journal): Humanities and Social Sciences, 4(1), 1161–1167. https://doi.org/10.33258/birci.v4i1.1727 Rahmawati, Y., Hadinugrahaningsih, T., Ridwan, A., Palimbunga, U. S., & Mardiah, A. (2021). Developing the critical thinking skills of vocational school students in electrochemistry through STEM - Project-based learning (STEM-PjBL). AIP Conference Proceedings, 2331. https://doi.org/10.1063/5.0041915 Risdianto, E. (2019). Analisis Pendidikan Indonesia di Era Revolusi Industri 4.0. Research Gate, April(January), 1–16. Rosyidah, N. D., Kusairi, S., & Taufiq, A. (2021). Kemampuan Berpikir Kritis Siswa melalui Model STEM PjBL disertai Penilaian Otentik pada Materi Fluida Statis. Jurnal Pendidikan: Teori, Penelitian, Dan Pengembangan, 5(10), 1422. https://doi.org/10.17977/jptpp.v5i10.14107 Setyowati*, Y., Kaniawati, I., Sriyati, S., Nurlaelah, E., & Hernani, H. (2022). The Development of Science Teaching Materials Based on the PjBL-STEM Model and ESD Approach on Environmental Pollution Materials. Jurnal IPA & Pembelajaran IPA, 6(1), 45–53. https://doi.org/10.24815/jipi.v6i1.23571 Sulisworo, D., Kaliappen, N., History, A., Winarti, W., Sulisworo, D., & Kaliappen, N. (2021). Indonesian Review of Physics (IRiP) Evaluation of STEM-Based Physics Learning on Students’ Critical Thinking Skills: A Systematic Literature Review Article Info ABSTRACT “Evaluation of STEM-Based Physics Learning on Students’’ Critical Thinking Skills: A S.” Indones. Rev. Phys, 4(2), 23–31. https://doi.org/10.12928/irip.v4i2.3814 Susilowati, Sajidan, & Ramli, M. (2017). Analisis keterampilan berpikir kritis siswa madrasah aliyah negeri di Kabupaten Magetan. Prosiding SNPS (Seminar Nasional Pendidikan Sains), 21(2000), 223–231. Thovawira, F. A., Safitri, I., Supartik, S., Sitompul, N. N. S., & Anggriyani, I. (2021). Systematic Literature Review: Implementasi Pendekatan Stem (Manfaat Dan Tantangan) Di Indonesia. HISTOGRAM: Jurnal Pendidikan Matematika, 4(2), 355–371. https://doi.org/10.31100/histogram.v4i2.682 Yanuar, Y. (2018). Unit Pembelajaran STEM . Bandung: South East Asia Ministry of Education Organization (SEAMEO) Regional Centre for Quality. Yustyan, S., Widodo, N., & Pantiwati, Y. (2016). Peningkatan Kemampuan Berpikir Kritis Dengan Pembelajaran Berbasis Scientific Approach Siswa Kelas X Sma Panjura Malang. JPBI (Jurnal Pendidikan Biologi Indonesia), 1(2), 240–254. https://doi.org/10.22219/jpbi.v1i2.3335