a © 2022 Indonesian Society for Science Educator 386 J.Sci.Learn.2022.5(3).386-397 Received: 03 July 2022 Revised: 16 October 2022 Published: 27 November 2022 The Impact of STEM Applications on Problem-Solving Skills of 4th-Grade Students Raşit Zengin1*, Tuğçe Kavak1, Gonca Keçeci1, Fikriye Kırbağ Zengin1 1Department of Mathematics and Science Education, Faculty of Education, Fırat University, Elazığ, Turkey *Corresponding author: rzengin@firat.edu.tr ABSTRACT This study examined the impact of STEM applications realized with 4th-grade elementary school students on their problem-solving skills. The study was conducted within the convergent parallel design framework, a mixed research method. The research was carried out with 42 4th-grade primary school students, 21 in the experimental group and 21 in the control group, studying in a public school in Turkey. In the study, which lasted for ten weeks, STEM applications were carried out with the students in the experimental group. The control group applied the current science curriculum without performing STEM applications. The research data consists of the Developing Problem Solving Inventory for Children at the Level of Primary Education (PSIC) and the semi-structured interview conducted with the students in the experimental group. Quantitative data were analyzed using an independent sample t-test, while qualitative data were analyzed with content analysis. When the PSIC post-test scores of the 4th- grade primary school students in the experimental and control groups were examined, a statistically significant difference was found in favor of the experimental group. According to the qualitative data analysis of the study, it was determined that the stude nts encountered various problems in the process and developed solutions for them. The students stated that using STEM activities in the classes is fun, so they learn more permanently and easily, and they want the classes to be delivered in this way from now on. Keywords STEM Education, Primary School 4th Grade Students, Problem Solving Skills 1. INTRODUCTION In the 21st century, the race between developed countries in the fields of invention, production, and technological development is accelerating. Thus, in such a competitive environment, governments have started to invest more in science, engineering, and innovative technologies (Ministry of National Education [MNE], 2018b). The age of technology, in which we live and where breathtaking developments are experienced, expects individuals to be able to think critically and analytically, solve daily life problems, be creative, research and question, and make effective decisions (Pekbay, 2017). For this reason, individuals now have new responsibilities. One of these responsibilities is to find solutions to the problems they will encounter in their daily life. With the increasing technology, people face more and more problems every day. In the past years, jobs that required muscle power and labor were expected from individuals, but now that there are machines that do these jobs, muscle power has been replaced by machines and technological devices. The new responsibility of individuals is to design, manufacture and market these machines. For these reasons, developed and developing countries aim to abandon education systems with only content teaching and turn to STEM education, which is a project-based interdisciplinary approach towards research, inquiry, invention, and production (Ministry of National Education [MNE], 2018b). STEM education; represents the relationship in which science, technology, mathematics, and engineering are intertwined and interlocked. STEM education aims to raise individuals willing to examine STEM-related issues, use scientific methods while doing research, and recognize the contemporary and cultural environment created by every discipline in STEM (Tiryaki & Adigüzel, 2021). Since students work interdisciplinary in STEM activities, they can produce practical solutions to problems and relate the relevant subject to daily life (Özçelik & Akgündüz, 2018). In addition, students gain multiple perspectives on problems as they actively participate in STEM activities (Bahar, Yener, Yılmaz, Hayrettin & Gürer, 2018). Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 387 J.Sci.Learn.2022.5(3).386-397 We may encounter several definitions when we examine the literature on problem definition. Dewey (1997) defined it as anything that interferes with the human mind and obscures and challenges beliefs. According to Karasar (2005), a problem is any desired problem to overcome. For the individual to want to overcome this difficulty, s/he must feel uncomfortable. Therefore, the problem can be defined as any situation that causes discomfort in the individual. When the diagnoses of the problem are analyzed, we can state that to call a case a problem. It must confuse people. The individual has not encountered this situation before, and this problem is new to them. For this reason, a situation that is a problem for one individual may not be a problem for another. For example, suppose the individual encounters a situation that requires a solution strategy and sees the situation but does not immediately think of the strategy. In that case, it can be said that there is a problem (Yenilmez, 2010). The ability to do what needs to be done when an individual does not know what to do is called a "problem-solving skill" (Üstün & Bozkurt, 2003). When we look at the Education system in Turkey, it is supported by experimental research results that different non-traditional problem-solving techniques improve students' problem-solving skills more compared to traditional problem-solving techniques (Gök & Sılay, 2008; Özkök, 2005). We can use STEM education to develop students' problem-solving skills with a non-traditional method because STEM education is an educational approach that provides students with problem-solving skills and enables them to think creatively (Sangngam, 2021). In this context, Turkey has aimed to introduce science, technology, engineering, and mathematics disciplines from an interdisciplinary perspective to students from an early age by switching to STEM education. At the same time, it is aimed to provide students with skills such as problem- solving, questioning, research, product development, and aesthetic perspective. Our students have the energy, talent, and opportunity to do several things. Therefore, we should increase these opportunities for them, encourage them to STEM education based on research and questioning, and make them realize their talents and achievements (Ministry of National Education [MNE], 2016; Asigigan & Samur, 2021). When the studies are examined, Doğan, Aydın & Kahraman (2020); Güven (2022); Hebebci (2019); Kartini, Widodo, Winarno & Astuti (2021); Kurt & Benzer (2020); Tran (2018); Yurttaş (2021) conducted STEM activities with middle school students and examined the effects of these activities on students' problem-solving skills. Akçay (2019); Öztürk & Çınar (2022); Şanlı (2021), and Yalçın & Erden (2021), on the other hand, examined the effects of STEM activities on preschool students' problem-solving skills. Acar (2020); Alan (2017); Öztürk & Yalçın (2020) looked at the impact of STEM activities on problem- solving skills with teachers and prospective teachers. Acar, Tertemiz & Taşdemir (2020); Asigigan & Samur (2021); Yıldız & Özdemir (2022) examined the effect of STEM activities on primary school students' problem-solving skills. The study was carried out to examine the effect of STEM activities on the problem-solving abilities of young students. 1.1. Problem of Research The innovation era we live in draws a roadmap for the future generation that will rule the country. Our children, called "Generation Z" or "digital settlers/digital natives", now prefer to interact and socialize through mobile communication devices rather than playing games on the street. Unlike us, it is accepted that children of this age are born with information and communication technologies. Therefore, to have a more significant say in global competition and to create a sustainable economy, well- planned STEM education should be offered to this generation, which makes up 17% of our population (Tekin Poyraz, 2018). When we look at the science curriculum, it is stated that the use of science and engineering applications in education has an important place in taking the necessary steps for our countries, such as competitiveness, socioeconomic development, scientific research, and technological development (Ministry of National Education [MNE], 2018a). When we look at the studies on STEM education, the number of studies conducted in Turkey has been increasing rapidly in recent years. However, it is seen that the majority of the studies are carried out with secondary school students, teacher candidates, and teachers. The study with primary school students is at a lesser level. With this study, it is thought that this need will be met to some extent. This study aimed to determine the effects of STEM education on the problem-solving skills of primary school 4th-grade students, and students opinions were obtained regarding the STEM applications conducted. 1.2. Research Focus STEM education is an integrated approach that adopts creative problem-solving techniques for our young scientists and engineer candidate students, who are the treasure of our future. STEM education is one of our most up-to-date training that aims to enable students to look at problems from an interdisciplinary perspective and gain the required knowledge and skills (Şahin, Ayar & Adıgüzel, 2014; Kartini, Widodo, Winarno & Astuti, 2021). For primary school students, the concept of engineering is a concept that has not yet been fully realized, does not attract much attention, and is ignored. Engineers have responsibilities in several technological fields, which the age of technology has made indispensable for our daily lives, and the need for this profession is increasing daily. However, younger students are not aware of the study fields and benefits of engineering. Therefore, it is very Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 388 J.Sci.Learn.2022.5(3).386-397 critical that we introduce this profession to the junior engineer candidates of the future, make it popular, and inspire the profession's importance in the first stages of education. Advocates of an integrated approach to STEM education in primary education argue that students' interest, success, and motivation can be increased, especially with real-world problems, which will help increase the number of students who want to pursue a career in STEM fields (Honey, Pearson, & Schweingruber, 2014). It is important to present various STEM applications as examples to our teachers who want to implement them and to bring these applications into the literature to make students interested in STEM fields and to disseminate and develop STEM education. 2. METHOD In this research, which is based on a mixed-method, a parallel pattern converging from mixed-method practices was used. The purpose of the contemporary parallel design is to collect quantitative and qualitative data to present the problem in the research in a comprehensive and detailed way. The main point is to bring together the mutually supportive aspects of the qualitative and quantitative methods and the non-overlapping elements. For this reason, if desired, data obtained by quantitative methods and data obtained by qualitative methods can be used to support each other (Creswell & Clark, 2015). 2.1. Working Process The research was carried out in a public school in Elazig in, Turkey. The study group the research consists of 42 primary school students from 4th-grade students enrolled at this primary school. Among these students, 21 were included in the experimental group, 13 of whom are girls and 8 of whom are boys. There were also 21 students in the control group, 9 of whom were girls and 12 of whom were boys. While determining the experimental and control groups, the year-end average scores of the previous year's branches were taken as a basis. The study covered ten weeks, and two weeks of the process were reserved for pre- test and post-test applications. While the science course curriculum and STEM applications were carried out with the students in the experimental group, the courses were taught only in line with the current science curriculum with the students in the control group. Figure 1 shows flow chart regarding the steps followed during the research Experimental and control groups were determined in the first week of the research, and pre-tests were applied to both groups. During the study process, the current program was adhered to in teaching the control group. In addition, the current curriculum, robotic studies, and fun science experiments were applied to the students in the experimental group by integrating them into STEM education. In the second week of the research, the "My Water is Multi-Colored" activity was held with the students. The students learned about the foods they use in their daily lives and what the concepts of acid and base contained in various materials are, and the concept of pH, which is even included in water bottles in their daily lives. Still, they don't have an idea about what it is. Cabbage juice was used as a separator, and different colors were obtained in cabbage juice by adding lemon, shampoo, vinegar, bleach, milk, juice, baking soda, shampoo, and soap to cabbage juice. The students classified the colors and concluded that one of the two groups formed was acid, and the other was base. Using microprocessor-supported laboratory sets, they found that pink and similar colors were between 0-7, while purple and blue color-like color dec received values between 7-14. Therefore, the pH December of acids and bases is inferred. In the third week of the research, the activity "We Design Cool Cars" was held with the students. Students were given materials such as pet bottles, bottle caps, balloons, tires, straws, garbage skewers, rulers, and cardboard, which are waste materials. They were asked to design a car by being told to use the materials they wanted. Then, the students were divided into two groups and created their cars using air thrust in a team effort within the specified time. As a result, they have gained awareness and made a design that air can be used as a force. Then, the two groups' students completed the cars they designed respectively. Figure 1 Flow chart regarding the steps followed during the research Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 389 J.Sci.Learn.2022.5(3).386-397 In the 4th week of the research, the "Whose raft is. Stronger" activity was held with the students. Students were asked about swimming and sinking events at the start of the event. After discussing swimming and sinking events, the question of why objects float or sink was asked, and swimming and sinking events were questioned with students. Finally, students learning about swimming and sinking were asked to design a raft. The students made their raft designs by working as a team within the specified time. In the 5th week, the activity "We Designed the 4th Bridge to Istanbul" was held with the students. Students were given various materials such as straws, cardboard, foam, sticks, two cars, play dough, and aluminum foil, and they were asked to design a bridge using the materials they wanted. The students shared ideas within the specified time and created a bridge. Two toy cars, one big and one small, were driven on both designed bridges, and the durability of the bridges were tested. In the 6th week of the practices, the activity "Balancing the Weights" was held. Students have designed leverage using lego robotic sets. The students were asked how these levers could be balanced by placing different loads on the two sides of the designed lifter. Students have found and interpreted how to balance a light load and a heavy load by trial and error. In the 7th week of the application, the "Energy from the Sun" event was held. Students have designed their solar-powered cars using lego robotic sets. The students raced their cars in the schoolyard with the opportunity to have sunny weather. Students have discovered that they can use the sun as an energy source. They deduced that this energy is an inexhaustible energy that exists in nature. Furthermore, they have concluded how this energy is stored. In the 8th week of the study, the "Don't Say that There Will Be No Wind-Powered Car" activity was held. Students have designed wind-powered cars using lego robotic sets. The students were asked whether they had seen the wind turbine before starting the exercise. After the students learned about using and storing wind energy, they designed their vehicles. The students in two groups then had the vehicles they designed raced. In the 9th week of the study, the "Eggs Don't Always Break" activity was carried out. Students were asked to design, so they were thrown from the specified height and did not break. The students designed according to their ideas, and the eggs were started to be thrown in the determined order. The last two unbroken eggs remain. In the 10th week of the study, post-tests were applied to the students in the experimental and control groups. In addition, interviews were conducted with the students in the experimental group about the practices carried out. Figure 2 depicts sample photos of STEM application in experimental groups 2.2.Data Collection Tools Quantitative and qualitative data tools were used to collect data for the study, which was carried out in line with the mixed-method design. For this purpose, the quantitative data collection tool of the analysis consisted of the Developing Problem Solving Inventory for Children at the Level of Primary Education (PSIC). The qualitative data was the semi-structured interview with the experimental group students. The "Problem Solving Inventory for Children" developed by Serin, Bulut-Serin & Saygılı (2010) was used in the research. The scale measures primary school students' perceptions of their problem-solving skills. The Figure 2 Sample photos of STEM application in experimental groups Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 390 J.Sci.Learn.2022.5(3).386-397 Cronbach's alpha reliability coefficient of the scale, which consists of a total of 24 items, was found to be 0.80. The researcher developed semi-structured interview questions and examined them by three faculty members at Firat University Faculty of Education, Mathematics and Science Education, Science Education Department. After they were structured in line with expert opinion, five 4th- grade students were asked to read the questions and indicate the points they did not understand. Then, the interview questions, which took their final form and consisted of 5 questions, were applied to the students in the experimental group to determine the feelings and opinions of the students in the experimental group where STEM applications were carried out. Interviews were conducted with 19 students in the experimental group for an average of 30 to 40 minutes. While giving sentences belonging to students in the interview analysis, real student names were not used, but pseudonyms were used. 2.3. Data Analysis SPSS 22 statistic package program was used to analyze quantitative data in the study. Descriptive statistical analysis was performed on the data belonging to each variable for the experimental and control groups. Standard deviation, arithmetic mean, skewness coefficient, kurtosis coefficient, and maximum and minimum values of the data belonging to the groups were calculated. The findings were used to understand the relations and distributions of the variables with each other and to control the assumptions before the analysis. Since the group size was smaller than 50 in the study, the Shapiro-Wilks Test was applied to examine the normality of the scores. An independent samples t-test was conducted to test the study's research question. Semi- structured interviews with primary school 4th-grade students were evaluated with content analysis. The basic process in content analysis is to gather similar data within the scope of specific themes and concepts to organize and interpret them in a way that readers can understand (Çepni, 2014). 3. RESULT AND DISCUSSION This study examined the impact of STEM applications on primary school 4th-grade students' problem-solving skills, and student opinions about the applications were analyzed. The students in the experimental group were offered STEM applications for eight weeks along with the science course curriculum. 3.1. Quantitative Data The average distribution of the problem-solving inventory pre-test scores for primary school children (Pre- PSIC) and the problem-solving inventory post-test for primary school children (Post-PSIC) applied to the study group was examined. The comparison of the test scores of the 4th-grade primary school students between the groups, the kurtosis and skewness values, standard deviation, Shapiro-Wilk values, and minimum and maximum values are given in Table 1. As seen in Table 1, before the application, the Pre-PSIC score mean of the experimental group (/=93.38) was very close to the Post-PSIC mean value of the control group ( =93.47). It is seen that the problem-solving inventory between groups was close to each other before the application. When both groups are examined, there are ( X difference=10.76) a points difference between the Pre-PSIC mean score ( X =93.38) and Post-PSIC mean score ( X =104.14) of the experimental group. In contrast, the difference is more negligible ( X difference=0.57) between the Pre-PSIC mean score ( X =93.47), and Post-PSIC means score ( X =94.04) of the control group. 3.2. Inferential Statistics Results an independent samples t-test was conducted to test the study's research question. The t-test results of the experimental and control groups before and after the application are given in Table 2. As shown in Table 2, there is no significant difference between the PSIC scores of the experimental and control groups before the application (t(40)=-.024, p>.05). X Table 1 Descriptive statistics of primary school 4th grade students' problem inventory test pretest and posttest scores for primary education level children." Tests Groups Gender N X SS Skewness Kurtosis Range Min Max Shapiro-wilk Pre-PSIC Experimental Female 13 97.53 12.29 -.055 -1.674 33 80 113 .088 Male 8 86.62 17.89 -.202 -1.94 45 64 109 .234 Total 21 93.38 15.23 -.491 -.719 49 64 113 .143 Control Female 9 95.15 7.96 -.852 .930 30 77 107 .488 Male 12 90.75 13.47 -.622 -1.29 37 69 106 .237 Total 21 93.47 10.31 -.958 .168 38 69 107 .043 Post-PSIC Experimental Female 13 104.76 13.05 .789 .523 39 78 117 .040 Male 8 103.12 10.5 -.136 -1.247 29 89 118 .613 Total 21 104.14 11.9 -.564 -.760 40 78 118 .046 Control Female 9 96.15 10.84 -.050 -.552 36 78 114 .801 Male 12 90.62 18.81 -.854 .450 59 56 115 .615 Total 21 94.04 14.21 -.909 1.307 59 56 115 .270 Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 391 J.Sci.Learn.2022.5(3).386-397 However, after the application, a significant difference was found between the PSIC scores of experimental group and control group (t (40)= 2.225, p<.05). This statistical difference was found in favor of experimental group 3.3. Qualitative Data Semi-structured interviews were conducted with the students in the experimental group to obtain their feelings and opinions about STEM activities. In terms of the ethics of the study, pseudonyms were used for the students. The researcher and three field specialists examined the data analyzed by content ana lysis. First, the codes were determined so that the meanings of the answers given by the students were not distorted. Then, after the regulations were identified, the answers were grouped within themselves and divided into the necessary categories. Finally, in the analysis conducted in the form of induction, the themes that would deliver the most general meaning were determined last. The students were asked, "Can you please indicate the contributions that the activities you participated in during the semester have made to you?" The table obtained as a result of the content analysis is given below. As a result of the analysis, three themes and categories were determined. These themes are "Contribution to Skills", "Contribution to Attitude," and "Contribution to Academic Achievement". Table 2 T-Test results of primary school 4th grade students' problem inventory pre-test and post-test scores for primary education level children N X SD df t p Pre-PISC Experimental 31 93.38 15.23 40 -.024 0,981 Control 31 93.47 10.31 Post-PSIC Experimental 31 104.14 11.9 40 2,225 0,032 Control 31 94.04 14.21 Table 3 Students' opinions on the contribution of the activities they attended during the semester Theme Category Code f % Contribution to Skills Scientific Process Skills Observation 18 94.7 Measuring 10 52.6 Classification 7 36.8 Using numbers 5 26.3 Inference 12 63.1 Experimentation 18 94.7 Deciding 12 63.1 21. Century Skills Problem-solving skill 6 31.6 Creativity and innovation 3 15.8 Communication and Collaboration 4 21.1 Contribution to Attitude Attitude toward Natural Sciences Increased interest in science 6 31.6 Love for science 1 5.3 Finding the class fun 1 5.3 Attitude towards Technology Liking technology 4 21.1 Increased interest in technology 6 31.6 Finding technology important 2 10.5 Attitude towards Engineering Interest in engineering 4 21.1 Love for design 2 10.5 Feeling like an engineer 8 42.1 Attitude towards Mathematics Increased interest in mathematics 4 21.1 Don't like math 2 10.5 Contribution to Academic Achievement Contribution to Course Achievement Science course success increase 9 47.4 Mathematics course success increase 5 26.3 My success has increased 2 10.5 Increase in Knowledge Better learning 4 21.1 Increase in knowledge 4 21.1 Learning to experiment 6 31.6 Contribution to Learning Active participation 2 10.5 Learning by doing 3 15.8 Learning to learn 1 5.3 Trial and error 2 10.5 Meaningful learning 2 10.5 Apply what is learned in real life 2 10.5 Intelligence increase 3 15.8 Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 392 J.Sci.Learn.2022.5(3).386-397 When Table 3 is analyzed, There are two categories under the "skills" theme, namely "Scientific Process Skills" and "21. Century Skills". Among the science process skills, 18 students stated observation skills, ten mentioned measurement skills, seven mentioned classification skills, five mentioned the ability to use numbers, 12 students stated deduction skills, 18 indicated experimentation skills, and 12 mentioned decision-making skills. The answers given by some of the students are given below: Pelin: It has contributed to my ability to observe because, in my experiments, I saw that I could understand and learn better by observing. Eren: It contributed to measurement; I learned that measurements are used in real life. I learned to make decisions while doing something. It contributed to my classification skills. I learned to separate things into different places. Selin: While observing, I observe the first and last states of objects as we do in experiments. Büşra: It has contributed to my decision-making skills because we have to make decisions while doing it ourselves. Also, it helped my ability to experiment because we did it ourselves. Ezgi: It contributed to my ability to experiment because I learned to test. I feel confident when experimenting because experimentation is an excellent thing. Hatice: It contributed to my ability to conclude; we would conclude all the activities and write them in our diaries. Tolga: It contributed to my classification skills. I used to classify materials in activities. Burcu: It has contributed to my ability to experiment because I think I can get better grades in science class by doing experiments and making better decisions about my classes by observing. The 21st-century skills category is based on the skills included in P21 (Partnership for 21st-century learning), used by several schools and educators in the United States (P21, 2017). There are three codes parallel to the skills of problem-solving, communication and cooperation, creativity, and innovation in the framework of P21. The answers given by some of the students are given below. Selin: I was undecided on some issues. I shared this with my group friends, who helped me too. Burcu: I struggled with how we combine things in applications. But I overcame it. Selin: I pushed myself to think, and we made new inventions. Büşra: Like engineers, we also produced something. Zuhal: We had difficulties because we did not work as a team. But in the end, we realized we were not working as a team, and we all worked together. Mehmet: We overcame the difficulties with teamwork. Ezgi: I faced difficulties in many spots, but I succeeded by gaining my trust. Sıdıka: I couldn't put the pieces together at first, but now I've learned. Eren: We invented bridges and designed eggs that don't crack, which is nice. Tolga: I realized that we could come together and do experiments. Ezgi: It taught me a lot. That is, it contributed a lot. For example, in engineering, I can build most things. I can make an underpass and an overpass. When Table 3 is analyzed, There are four categories under the attitude theme. These are attitude towards science, attitude towards technology, attitude towards engineering, and attitude towards mathematics. In the category of attitude towards science, it was observed that six students' interest in science increased, one student liked science and technology, and one student found the class enjoyable. The answers given by some of the students are given below: Mehmet: My interest in science has increased. Sıdıka: These activities made me love science and technology. Burcu: It increased my interest in science class. Selin: Doing the activities increased my interest in science classes. In the category of attitude towards technology, it is seen that four students like technology, six students are more interested in technology, and two students find technology important. The answers given by some of the students are given below: Eren: I started to find technology important with these activities. Pelin: My interest in technology has increased, and my life will be easier with technology. Burçak: We can use technology daily, and our interest in technology increases. Tolga: I was interested in both technology and its benefits. Alperen: It developed my interest in technology because we made a solar-powered car, which really attracted my attention. In the category of attitude toward engineering, it is seen that four students said that their interest in engineering increased, two students liked to design, and eight students felt like engineers. The answers given by some of the students are given below: Alperen: We built leverage. I felt like an engineer when I built leverage. Burçak: My interest in engineering is increasing. Ezgi: I loved designing. Kaan: We did a lot of things like engineers. Tolga: I felt like an engineer during the events. Pelin: My interest in engineering grew. Regarding attitude towards mathematics, four students stated that their interest in mathematics increased, and two indicated that they liked mathematics. The answers given by some of the students are given below: Sıdıka: The activities contributed to my mathematics class, and my interest increased. Emre: I loved math very much with these activities. When Table 3 is analyzed, Academic success is studied under three categories: contribution to course achievement, increase in knowledge, and learning assistance. In the dimension of Contribution to Course Achievement, nine students stated that they experienced an Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 393 J.Sci.Learn.2022.5(3).386-397 increase in their science course achievement, and five students stated that they experienced an increase in mathematics course achievement. In addition, two students stated that they experienced an increase in their general achievement. The answers given by some of the students are given below: Selin: My science and technology class knowledge has been updated, and my success level has increased. Hatice: These activities increased my math achievement. Burcu: I think I can get better grades in science class. Tuba: In science class, the same topics come up in our books, and you know this beforehand. It also increased my success in mathematics because I learned to measure. Pelin: The activities improved my measurements and estimations in mathematics, and my success level increased in this way. Alperen: I am learning and succeeding in something new in science class. For example, I learned about objects that sink in and float on water, and I will use this knowledge in science class. In the category of increase in knowledge, it was observed that six students answered that they learned to experiment, four responded that they learned better, and four answered that their knowledge increased. Some of the students' answers are given below: Kaan: Thanks to the activities, my knowledge increased. Eren: I learned to experiment. Zuhal: We learned better things. Gamze: The activities contributed a lot. For example, I learned to experiment. Ezgi: I learned to experiment. I feel confident when experimenting. I learned many things, and my knowledge increased. Burçak: We learn about both experiments and science. In the category of contributing to learning, two students stated that they contributed actively to participating, three students mentioned learning by doing, one student mentioned learning to learn, two students mentioned trial and error, two students mentioned meaningful learning, two students mentioned learning by applying what was learned in real life, and three students mentioned increase in their intelligence. The answers given by some of the students are given below: Pelin: It helped me a lot. I found ways to learn better. Hatice: The activities contributed helped me understand better. Tuba: The activities contributed to me because we both did and learned. Zuhal: I used to say there is no such thing as an uncrackable egg, but we tried it and saw that it is possible to make uncrackable eggs. The question "Do you think that there is a connection between Science-Technology-Engineering-Mathematics as a result of the applications carried out?" was asked to the students. The frequency and percentage distribution of their answers to the posed question are given in Table 4. In response to the question "Do you think that there is a connection between science-technology-engineering- mathematics as a result of the practices performed", it was concluded that 16 students thought that at least one or more of these areas were involved in the activities carried out, and were related. In comparison, three students did not find a connection. Among the students who thought that there was a connection, it was concluded that two students believed that the realized applications were only related to science, one student thought that there was a connection between the fields of science and technology, three students thought that engineering and maths were related, five students believed that science, technology, and engineering fields were interrelated, one student believed that there was a connection between science, engineering, and maths, one student thought that science, technology, and maths were related, and three students believed that the fields of science, technology, engineering, and maths were interrelated. Answers given by some of the students are provided below. Burcu: Yes, I think there is a connection. While doing the rafting activity, we learned that water is buoyant in science, and we learned counting in mathematics. I also liked to produce in engineering. Alperen: For example, I think I am dealing with technology while making a wind-powered car. When we put sinking and non- sinking objects in a basin full of water, I learn about science. For example, when I was building a bridge, I had a dream as if I was an engineer. Selin: I think so. For example, science and technology. We did experiments on science and improved our technology. There is also a relationship between mathematics and engineering, and we do it in some ways in mathematics. In engineering, we invent, as in experiments. Table 4 Frequency and percentage distribution of students' responses to whether there is a connection between science, technology, engineering, and mathematics as a result of the applications Connection Dimension Name of Discipline f % There is connection Only Science 2 10.5 Science-Technology 1 5.3 Engineering-Math 3 15.8 Science-Technology-Engineering 5 26.3 Science-Engineering-Mathematics 1 5.3 Science-Technology-Math 1 5.3 Science-Technology-Engineering-Math 3 15.8 There is no connection 3 15.8 Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 394 J.Sci.Learn.2022.5(3).386-397 Gamze: We designed a solar-powered car with technology. I felt like an engineer. Burçak: I think it's just science. Because the activities we do look like science. Büşra: I think there is a relationship. For example, I learned something about science while making a raft. While building the raft, we did it with math information. I did technology while creating a raft. Tolga: There is no connection because there is no relation between them. Table 5 shows the frequency and percentage distribution of the answers given by the students to the question, "Would you explain if you had difficulties during the implementations?" In response to the question, "Would you please explain if you had difficulties during the implementations," which was asked the students, ten students stated that they had difficulties. In comparison, nine students indicated that they did not have any problems. Among the students who had difficulties, it was seen that five students answered that they had difficulties in designing, one student mentioned assembling Lego pieces, one student mentioned classifying, two students mentioned teamwork, and one student mentioned using the activity materials. The answers given by some of the students are given below: Pelin: I did not have any difficulties; even if I did, I managed to cope with them. Sıdıka: I had a hard time designing a bridge at first, but then I learned. Mehmet: I had a hard time putting the Lego pieces together because if we put the Lego in the wrong place, that experiment wouldn't work. Büşra: I have not had any difficulties so far. Tuba: Yes, I did. For example, I had a little difficulty making the edges of the raft. Table 6 shows the frequency and percentage distribution of the answers given to the students for the question, "Which activity did you enjoy the most from the practices? Can you explain?" To the question posed to the students, "Which activity did you enjoy the most from the practices carried out, please explain", seven students cited "Energy from the Sun". Among the students who enjoyed this activity, 3 of them found it interesting, two said they saw it for the first time, one One student liked the "Don't say you can't have a wind-powered car" activity because it caught their attention. Six students said they enjoyed the "Eggs don't always crack" activity. Three students who chose this activity stated that they liked designing, one said they wanted it, and two said eggs always cracked. On the other hand, two students expressed the activity "We are building the 4th Bridge to Istanbul" with the explanation that "it caught their attention". Furthermore, one student stated, "My water is very colorful" activity, and another said that "the color change caught my attention". On the other hand, one student expressed the activity "Whose raft is more durable" as the activity they liked the Table 5 Frequency and percentage distribution of the answers regarding whether the students experience difficulties during the applications Student status Codes f % Having difficulty Designing 5 26.3 Assembling Lego pieces 1 5.3 Making a classification 1 5.3 In teamwork 2 10.5 Use of event materials 1 5.3 Not having difficulty I did not experience any difficulties 9 47.4 Table 6 Frequency and percentage distribution of students' responses to the applications they liked most Name of the Activity % Codes f % The energy coming from the sun 36.8 Interesting 3 15.8 I saw it for the first time 2 10.5 I was intrigued 1 5.3 It works differently from a normal car 1 5.3 Don't say you can't have a wind-powered car 5.3 I was intrigued 1 5.3 Eggs don't always crack 31.6 I like to design 3 15.8 I loved 1 5.3 I thought eggs always cracked 2 10.5 We are building the 4th bridge to Istanbul 10.5 I was intrigued 2 10.5 My water is very colorful 10.5 It felt so weird 1 5.3 The color change caught my attention 1 5.3 Whose raft is more durable 5.3 I was intrigued 1 5.3 Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 395 J.Sci.Learn.2022.5(3).386-397 most because it attracted their attention. The answers given by some of the students are given below: Tolga: Solar-powered car, because the solar-powered car is an exciting thing. Zuhal: An uncrackable egg, because I used to say there can never be an uncrackable egg, but we tried and saw that it is indeed possible to make uncrackable eggs. Ahmet: Color-changing water because it was so weird. Eren: I loved the uncrackable egg activity, and I liked to design. Burcu: Whose raft is more durable was the activity I liked the most because it caught my attention. Table 7 shows the frequency and percentage distribution of the answers the 4th-grade primary school students gave to the question, "Would you like to receive science classes with such activities or similar activities? Why?" All students answered "Yes, I would like" to the question "Would you like the science course to be taught with such or similar activities? Why?" Seven students said, "the activities are so nice, and they contribute," seven students said, "I am learning by having fun", one student said, "My handicraft is improving", and one student said, "I learn permanently", one student said "I learn easily" and two students said, "It contributes to the science class". The answers given by some of the students are given below: Alperen: I would because while making the applications, we both learn something and transfer that information to the science class. Tuba: Yes, I would because our classes are so much fun this way. Burcu: Yes, because such activities can contribute to further developing our handicrafts. Mehmet: Yes, I would, because it contributed and the classes are so good. Pelin: I would like to continue doing activities like this because we had more fun, which helped us learn better. Ezgi: Yes, I would because we both had fun and learned some things. It gets better and better this way. Sıdıka: I would like to learn more efficiently, and I do not forget. In this study, which was carried out with 4th-grade primary school students, the effect of STEM applications on students' problem-solving skills was examined, and student opinions were obtained. It was observed that the problem-solving skills of the 4th-grade primary school students in the experimental group where STEM applications were carried out were significantly higher than those of the 4th-grade primary school students in the control group where STEM applications were not performed. Based on this result, it can be stated that STEM applications effectively improve the problem-solving skills of 4th-grade primary school students. As a result of the findings obtained through the analysis of the interviews with 4th-grade primary school students after the applications, it was seen that the primary school students encountered various problems during the applications, defined the problem, tried to develop a solution, and finally solved the issues during the activities. When the relevant literature is examined, Pekbay (2017) conducted STEM activities with secondary school 7th- grade students and examined the effects on students' problem-solving skills based on daily life. In the study, it was concluded that STEM activities were influential in the development of students' problem-solving skills. Ceylan (2014) examined the effect on students' problem-solving skills by applying an instructional design based on STEM education on acids and bases. The author concluded that the students in the experimental group were more successful in problem-solving skills compared to the control group. Alan, Zengin, & Keçeci (2019) conducted a STEM application to support the integrated teaching knowledge of pre-service science teachers and examined the impact on students' problem-solving skills. It was concluded that the STEM applications were influential in developing problem-solving skills in the experimental group pre-service teachers compared to the control group pre-service teachers. In the literature, there are studies on the problem-solving abilities of primary school 4th-grade students in science classes. Erden &Yalcin (2021) studied the effect of STEM education on preschool students' creativity and problem-solving skills by applying STEM education prepared according to the design thinking model. As a result of the analysis, it was concluded that there was a significant and permanent increase in the creativity and problem-solving skills of the experimental group. Demir (2018) examined whether there is a relationship between primary school 4th-grade students' perceived problem-solving skills in science classes and their routine and non-routine problem-solving skills. In their research, English, King & Smeed (2017) asked sixth graders to build earthquake-resistant structures to follow the engineering design processes. Table 7 Frequency and percentage distribution of students' views and opinions on the realization of science classes with STEM activities Codes f % Yes, I would like The events are very friendly, and they make a contribution 7 36.8 I am learning by having fun 7 36.8 My handicraft is improving 1 5.3 I learn permanently 1 5.3 I learn easily 1 5.3 It contributes to science class 2 10.5 Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 396 J.Sci.Learn.2022.5(3).386-397 When the study results were examined, the students stated that their skills in solving engineering problems improved, and the intelligence of STEM basic concepts was ensured. Parno, Yuliati, & Ni'mah (2019), in their research, aimed to reveal the influence of Problem Based Learning-Science Technology Engineering Mathematics (PBL-STEM) on students' problem-solving skills in the topic of optical instruments. The result showed that the learning model influenced students' problem-solving skills. Experiment I and II groups, obtained an increase in problem-solving skills at a level higher than the control group. As a result of the research found a significant relationship between students' ability to solve routine and non-routine problems and their perceived problem-solving skills (Acar, Tertemiz & Taşdemir, 2020). The problem- solving abilities result obtained within the scope of the study coincide with the problem-solving skills results of secondary school students of STEM education. It is also compatible with the problem-solving abilities performed by primary school 4th-grade students. It is aimed to raise our students, entrusted to us today, who will be the adults of the future, as individuals with 21st- century skills who produce, contribute to economic and social developments, and follow the requirements of the world and the age. Moreover, the need for students who think, research, question, and invent is increasing daily. Therefore, countries are searching for an education model that will prepare students for life as responsible individuals with high problem-solving, critical thinking, and decision- making skills. For this reason, STEM education, which enables students to transform the theoretical knowledge they have learned in science, technology, engineering, and mathematics disciplines into practice, innovative inventions, and products, is being included in the curriculum of many countries in the world (Ministry of National Education [MNE], 2018b). 4. CONCLUSION When we look at the studies on STEM education, the number of studies conducted in our country has been increasing rapidly in recent years. However, most studies are conducted with secondary school students, pre-service teachers, and teachers. Due to the scarcity of studies on primary school students, it is recommended that researchers conduct studies with primary school students to overcome this deficiency. For STEM educators who will carry out STEM applications, it is recommended to prepare booklets containing STEM activities following student levels. For a more effective STEM education to be implemented, schools should be arranged to provide conditions favorable for STEM education. It is recommended that STEM activities be carried out with STEM educators in the design-skill workshops, which are planned to be opened in the 2023 vision REFERENCES Acar, D. (2020). The role of STEM awareness in predicting the contribution of teachers' problem solving skills and behaviors to the development of creative thinking. Academia Journal of Educational Research, 5(1), 77-89. https://dergipark.org.tr/en/download/article-file/1094399 Acar, D., Tertemiz, N. & Taşdemir, A. (2020). The relationship between mathematics and science problem solving skills and achievements of students who were being educated with STEM. Bartın University Journal of Educational Research, 3(2), 12-23. https://dergipark.org.tr/en/pub/bujer/issue/54228/712374 Akçay, B. (2019). STEM etkinliklerinin anaokuluna devam eden 6 yaş çocukların problem çözme becerilerine etkisi [The effects of STEM activiıes on the problem solving skills of 6 aged preschool children] (Master's Thesis). Yildiz Technical University, Turkey. Alan, B. (2017). Supporting preservice science teachers' integrated teaching knowledge: Preparation training practices to STEM. [Unpublished master’s thesis]. Fırat University, Turkey Alan, B., Zengin, F. K., & Keçeci, G. (2019). Using STEM Applications for Supporting Integrated Teaching Knowledge of Pre-Service Science Teachers. Journal of Baltic Science Education, 18(2), 158-170. https://doi.org/10.33225/jbse/19.18.06 Asigigan, S. I. & Samur, Y. (2021). The effect of gamified STEM practices on students' intrinsic motivation, critical thinking disposition levels, and perception of problem-solving skills. International Journal of Education in Mathematics, Science, and Technology (IJEMST), 9(2), 332-352. https://doi.org/10.46328/ijemst.1157 Bahar, M., Yener, D., Yılmaz, M., Hayrettin, E. & Gürer, F. (2018). The changes of standards in the 2018 science curriculum and STEM integration. Abant İzzet Baysal University Journal of Education Faculty, 18(2), 702-735. https://doi.org/10.17240/aibuefd.2018..-412111 Ceylan, S. (2014). A study for preparing an instructional design based on science, technology, engineering and mathematics (STEM) approach on the topic of acids and bases at secondary school science course. [Unpublished Master Thesis]. Uludağ University Education Science Institute, Bursa, Turkey. Creswell, J. W., & Clark, V. L. P. (2015). Pesquisa de Métodos Mistos-: Série Métodos de Pesquisa [Mixed Method Research-: Research Methods Series.] Penso Editora. Çepni, S. (2014). Introduction to Research and Project Work (7. Edition). Trabzon: Celepler Printing Press. Demir, D. (2018). The relationship between students' association with daily life and perceived problem solving skills and their routine and non-routine problem solving skills (Master's Thesis), Kocaeli University, Turkey. Dewey, J. (1997). How We Think? New York: Prometheus Books. Doğan, A., Aydın E & Kahraman E. (2020). The effect of STEM activity applications on problem solving skill perceptions of middle school students. ESTUDAM Journal of Education, 5(2), 123-144. https://dergipark.org.tr/en/download/article-file/1197604 English, L.D., King, D. & Smeed, J. (2017). Advancing integrated STEM learning through engineering design: sixth grade students" design and construction of earthquake resistant buildings. The Journal of Educational Research, 110(3), 255-271. https://doi.org/10.1080/00220671.2016.1264053 Erden, S. & Yalçın, V. (2021). Investigation of the effect of preschool STEM activities prepared according to the problem based learning approach on children's problem-solving skills. Trakya Journal of Education, 11(3), 1239-1250. doi: 10.24315/tred.789922 Gök, R. & Sılay, I. (2008). The effects of problem-solving strategies on students' achievement. on the cooperative learning groups in physics teaching. Hacettepe Universty Journal of Education, 34, 116-126. http://www.efdergi.hacettepe.edu.tr/shw_artcl-519.html Güven, H. (2022). The effect of STEM and STEM-based robotic activities on the problem solving and metacognitive skills of secondary school students (Master's Thesis). Erzincan Binali Yıldırım University, Turkey Hebebci, M. T. (2019). The impacts of science, technology, engineering and mathematics applications on middle school students' academic achievement, https://dx.doi.org/10.33225/jbse/19.18.06 https://doi.org/10.46328/ijemst.1157 https://doi.org/10.17240/aibuefd.2018..-412111 Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.48182 397 J.Sci.Learn.2022.5(3).386-397 scientific creativity and attitudes (PhD dissertation). Necmettin Erbakan University, Turkey. Honey, M., Pearson, G. & Schweingruber, H. (Eds). National academy of engineering and national research council (2014). STEM Integration in K-12 Education: Status, Prospects, and An Agenda for Research. Washington D.C. : The National Academies Press. Karasar, N. (2005) Scientific Research Methods (15. Edition). Ankara: Nobel Publication Distribution. Kartini, F. S., Widodo, A., Winarno, N., & Astuti, L. (2021). Promoting student's problem-solving skills through STEM project-based learning in earth layer and disasters topic. Journal of Science Learning, 4(3), 257-266. https://doi.org/10.17509/jsl.v4i3.27555 Kurt, M. & Benzer S. (2020). An investigation on the effect of STEM practices on sixth grade students' academic achievement, problem solving skills, and attitudes towards STEM. Journal of Science Learning, 3(2), 79-88. doi: 10.17509/jsl.v3i2.21419 Ministry of National Education -MNE (2018b). STEM Education Teacher's Handbook. Ankara: Ministry of National Education, General Directorate of Innovation and Educational Technologies (YEĞİTEK). Ministry of National Education-MNE (2016). STEM Education Report. Ankara: Ministry of National Education, General Directorate of Innovation and Educational Technologies (YEĞİTEK). Ministry of National Education-MNE (2018a). Science Course (Primary and Secondary Schools 3, 4, 5, 6, 7 and 8th Grades) Curriculum. National Education Press, Ankara. Özçelik, A. & Akgündüz, D. (2018). Evaluation of gifted/talented students' out-of-school STEM education. Trakya University Journal of Education Faculty, 8(2), 334-351. doi: 10.24315/trkefd.331579 Özkök, A. (2005). Effects of interdisciplinary creative problem solving teaching program on creative problem solving skills. Hacettepe Universty Journal of Education, 28(28), 159-167. http://www.efdergi.hacettepe.edu.tr/shw_artcl-784.html. Öztürk, S. C. & Yalçın, S.A. (2020). The effect of STEM education on pre-service science teachers' problem solving skills. Turkish Studies Education, 15(4), 2893-2915. https://dx.doi.org/10.47423/TurkishStudies.43707 Öztürk, Z. D. & Çınar, S. (2022). The effect of engıneerıng desıgn based STEM educatıon on the problem-solvıng skılls of pre-school students. Trakya Journal of Education, 12(1), 34-56. doi: 10.24315/tred.868414 Pekbay, C. (2017). Effects of science technology engineering and mathematics activities on middle school students. (Master's Thesis), Hacettepe University, Ankara. Sangngam, S. (2021). The development of early childhood students' creative thinking problem solving abilities through STEM Education learning activities. In Journal of Physics: Conference Series (Vol. 1835, No. 1, p. 012008). IOP Publishing. Serin, O., Bulut-Serin, N., & Saygılı, G. (2010). Developing problem solving inventory for children at the level of primary education (PSIC). Elementary Education Online, 9(2), 446-458. https://dergipark.org.tr/en/download/article-file/90752 Şahin, A., Ayar, M. C., & Adıgüzel, T. (2014). STEM related after-school program activities and associated outcomes on student learning. Education Sciences Theory & Practice, 14(1), 297-322. doi: 10.12738/estp.2014.1.1876 Şanlı, Z. S. (2021). The effects of STEM activities on creative thinking and problem solving skills of 60-72 month-children's in early childhood (PhD dissertation). Hacettepe University, Turkey. Tekin Poyraz, G. (2018). Examination of STEM education in Kayseri province and feasibility of distance STEM education. (Master's Thesis), Anadolu Universty, Turkey. Tiryaki, A., & Adigüzel, S. (2021). The Effect of STEM-Based Robotic Applications on the Creativity and Attitude of Students. Journal of science learning, 4(3), 288-297. doi:10.17509/jsl.v4i3.29683 Tran, Y. (2018). Computer programming effects in elementary: perceptions and career aspirations in STEM. Technology, Knowledge and Learning, 23(2), 273-29. https://doi.org/10.1007/s10758-018- 9358-z Üstün, A., & Bozkurt, E. (2003). Some occupational factors affecting the problem solving skills of primary school principals according to their perceptions of themselves. Kastamonu Education Journal, 11(1), 13-20. https://akademiye.org/tr/wp- content/uploads/2021/05/TURK-KULTURUNDE-VE- AZERBAYCAN-DESTANLARINDA-AT.pdf#page=15 Yalçın, V. & Erden, Ş. (2021). The effect of STEM activities prepared according to the design thinking model on preschool children's creativity and problem-solving skills. Thinking Skills and Creativity, (41), 1-14.doi: https://doi.org/10.1016/j.tsc.2021.100864 Yenilmez, K. (2010). Primary school students' determination levels about problem types. The Journal of SAU Education Faculty, 19, 124- 137. https://dergipark.org.tr/en/download/article-file/115623 Yıldız, S., & Özdemir, M. (2022) İlkokul öğrencilerinin problem çözme becerilerinin ve steme yönelik tutumlarının incelenmesi [examınıng of prımary school students’ problem-solvıng skılls and attıtudes towards stem]. Milli Eğitim Dergisi, 51(234), 987-1010. doi: 10.37669/milliegitim.837070 Yuliati, L., & Ni'Mah, B. Q. A. (2019, February). The influence of PBL- STEM on students' problem-solving skills in the topic of optical instruments. In Journal of Physics: Conference Series (Vol. 1171, No. 1, p. 012013). IOP Publishing. Yurttaş, Ş. (2021). The effect of group engineering design-based robotics applications on students' daily life-based problem solving skills (Master's Thesis). Bursa Uludağ University, Turkey. https://doi.org/10.17509/jsl.v4i3.27555 http://www.efdergi.hacettepe.edu.tr/shw_artcl-784.html https://doi.org/10.1007/s10758-018-9358-z https://doi.org/10.1007/s10758-018-9358-z