a © 2022 Indonesian Society for Science Educator 509 J.Sci.Learn.2022.5(3).509-519 Received: 18 April 2022 Revised: 1 June 2022 Published: 27 November 2022 The Investigation of the Research on Out-of-School Learning Activities in Turkey: A Systematic Review Mustafa Metin1*, Emir Oker1, Aslı Saylan Kirmizigül1 1Faculty of Education, Erciyes University, Kayseri, Turkey *Corresponding author: mustafametin@erciyes.edu.tr ABSTRACT This research aims to analyze the studies on out-of-school learning activities in Turkey between 2000 and 2020. For this purpose, 303 studies were selected, of which 211 articles, 73 master's thesis, and 19 doctoral dissertations, with the phrase "out-of-school learning" in the title and keywords. A systematic literature review was conducted in the research. The data were entered into the Content Analysis Monitoring Form, and the analyses were made on the Microsoft Excel program. In line with the findings obtained, it was determined that most of the studies on out-of-school learning activities were carried out in 2020. In these studies, it was seen that it was mainly aimed to determine the opinions of teachers or students about out-of-school learning activities, the sample was selected in the range of 21-40 people, and the most selected sample group was secondary school students. It was also determined that the studies were mostly done in the science discipline and that the case study from qualitative and experimental research design from quantitative methods was used. When all the findings were evaluated, since there are not many studies on the effect of out-of-school learning activities on anxiety, motivation, and interest, it is recommended to carry out studies in this area and to increase the studies to be done towards high school, preschool, primary school students. Keywords Out-of-School Learning, Systematic Review, Content Analyses 1. INTRODUCTION The age we live in has been defined as the age in which information is rapidly renewed and produced" and one of the characteristics that individuals should have been expressed as accessing information, using, and producing information (Ministry of National Education [MoNE], 2005). In this context, individuals are expected to realize the change, produce information by interpreting rather than accepting it as it is and prepare themselves for innovations and development. Therefore, the education given to the students plays a vital role in achieving the typical characteristics of individuals. Education continues throughout the individual's life and includes formal, non-formal, and informal learning activities (Lockhart, 2016; UNESCO, UNDP, UNFPA, UNHCR, UNICEF, UN WOMEN, …, ILO, 2015). Formal learning takes place with the effort of acquiring the pre-determined knowledge and skills of the individual under the umbrella of an educational institution within a certain period and plan (Güven, 2010). However, learning may not take place only in educational institutions. Learning environments outside educational institutions are informal and non-formal (Eshach, 2007). Informal learning is lifelong learning that takes place randomly in daily life without being tied to a specific program and authority (Cross, 2007; Metin & Özcan, 2015; Öner & Öztürk, 2019). Informal environments are the areas that do not have institutional features and that we can visit at any time (Tal & Morag, 2009). These learning environments are unplanned and haphazard. Here, the individual unwittingly acquires new information due to the situation s/he encounters and the interaction with the members of the group that s/he is part of (Fidan, 2012). Informal learning can occur when an individual discovers new things in the park, on the street, in the cinema hall, while watching TV, or chatting with friends (Bozdoğan, 2007; Metin & Özcan, 2015). In addition, mobile devices, home environments, e- learning, and web 2.0 applications are examples of informal learning environments (Bell, Lewenstein, Shouse, & Feder, 2009; Eshach, 2007). Non-formal learning is used by mailto:mustafametin@erciyes.edu.tr Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 510 J.Sci.Learn.2022.5(3).509-519 different names such as out-of-doors learning, free-choice learning, out-of-school learning, out-of-class experiences, outdoor learning, and out-of-school experiences in the literature (Ayotte-Beaudet, Potvin, Lapierre, & Glackin, 2017; Strauss & Terenzini, 2007; Şimşek & Kaymakçı, 2015). The most widely used definition is out-of-school learning (Bozdoğan, 2007). Out-of-school learning is systematic, planned, and structured learning activities outside formal learning environments (Bilir, 2007). One of the most critical differences between out-of-school and informal learning is that informal learning is unplanned, while non-formal learning is carried out within a specific plan (Bozdoğan, 2007; Eshach, 2007; Şen, 2019). Therefore, out-of-school learning is located at the intersection of formal and informal learning and acts as a bridge between them. Furthermore, it was stated that out-of-school learning consists of environmental education, outdoor activities, and personal and social development (Higgins, Loynes, & Crowther, 1997). Therefore, it can be said that out-of- school learning is related to many disciplines (Erten & Taşçı, 2016). Out-of-school learning, which is related to many disciplines, includes learning activities by visiting places such as a zoo, botanical garden, museum, science center, national park, planetarium, excursions and nature activities, interactive exhibitions, aquarium, and industrial enterprise within the framework of a specific plan (Eshach, 2007; Bostan Sarıoğlan & Küçüközer, 2017; Bozdoğan, 2007; Laçin Şimşek, 2011). Similarly, according to Binbaşıoğlu (2000), all out-of-school learning activities, especially school trips, can be examples of out-of-school learning because they are planned and controlled by the teacher. The common point in the statements of many researchers is the emphasis that out-of-school learning is not just a trip but that these trips are carried out within a specific plan. In the literature, it was stated that out-of-school learning activities carried out within a specific plan increase the students' academic achievement, motivation, and attitudes towards the lessons, enable students to learn by having fun, develop their sense of responsibility and curiosity, and positively affect their communication with their friends and provide solutions to the daily life problems (Bozdoğan, 2008; Braund & Reiss, 2006; Dori & Tal, 2000; Eshach, 2007; Higgins et al., 1997; Melber & Abraham, 1999; Ramey-Gassert, 1997; Strauss & Terenzini, 2007). Moreover, it was stated that out-of- school learning develops students' interpersonal, social, and leadership skills, provides students with individual and team learning skills, contributes to the self, social and moral development of the students, and allows them to make efficient use of their spare time and gain a democratic attitude (Eshach, 2007; Higgins et al., 1997; Karademir, 2013). There are also some limitations of out-of-school learning. Some limitations were revealed in the literature, such as the inability to achieve productive results with out- of-school learning in cases where the student's prior knowledge and necessary readiness are not available, the limited participation of students with low socioeconomic status in such activities, and the teachers' unwillingness to organize out-of-school activities without financial support (Ay, Anagün, & Demir, 2015; Bozdoğan 2007; Karademir 2013; Karamustafaoğlu, Ayvalı, & Ocak 2018; Türkmen, 2018). In addition, there are disadvantages such as; out-of- school learning activities cost would be high if not carried out within a specific plan framework, it would take more time, and it would be challenging to ensure the students' safety. Problems would also arise in concentrating the students' attention on the activities and in classroom control, making the desired learning quality unachieved (Ay et al., 2015; Bostan Sarıoğlan & Küçüközer, 2017; Karamustafaoğlu et al., 2018; Kubat, 2018; Türkmen, 2015; 2018). The countries considering that the benefits of the out- of-school learning environment are more than its limitations are trying to increase and diversify the number of applications in this field daily. In recent years, the number of environments such as museums, planetariums, zoos, botanical gardens, science centers, and aquariums has increased rapidly in our country, and it has come to the fore to use these environments for educational purposes, considering that these environments will create rich opportunities for students (Laçin Şimşek, 2011). In the Out-of-School Learning Environments Guide published by the Ministry of National Education in 2019, out-of- school learning was defined, and detailed information was given on how to evaluate out-of-school learning environments. Following the Ministry of National Education's guide, Provincial Directorates of National Education also published guides introducing out-of-school learning environments in their provinces and organized in- service training for teachers. Moreover, targets for out-of- school learning environments were determined in the 2023 Education Vision of the Ministry of National Education (MoNE, 2018a). The 2023 Educational Vision emphasizes that the cooperation between schools, science centers, museums, art centers, technology centers, and universities will be increased. The "Out-of-school learning environments" professional knowledge elective course was added to the program of education faculties in 2018 by the Council of Higher Education. In addition, out-of-school learning environments course was added as compulsory to some departments, such as science and primary school mathematics teaching (Council of Higher Education [CoHE], 2018). The number of out-of-school learning environments in our country is increasing rapidly, and the studies of the Ministry of National Education on out-of-school learning Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 511 J.Sci.Learn.2022.5(3).509-519 have begun to pay off. More and more students have started to visit out-of-school learning environments, and a significant increase has been observed in school trips to these environments. Accordingly, there has been an increase in studies on out-of-school learning. When the literature is examined, it is seen that there are studies that investigate the opinions of students, pre- service teachers and teachers about out-of-school learning (Akman, Özen Altınkaynak, Ertürk Kara, & Can Gül, 2015; Avcı & Gümüş, 2019; Ay et al., 2015; Aydemir & Toker Gökçe, 2016; Bakioğlu & Karamustafaoğlu, 2020; Batman, 2020; Bozdoğan, 2017; Büyükkaynak, Ok, & Aslan, 2016; Güngör & Göloğlu Demir, 2022; Uludag, 2021), and investigate the effects of out-of-school learning on students' academic achievement (Akça, Balkan Kıyıcı, & Yıldız, 2017; Bolat, Karamustafaoğlu, & Karamustafaoğlu, 2020; Bozdoğan & Kavcı, 2016; Bülbül, 2018; Karakaş Özür & Şahin, 2017; Küçük & Yıldırım, 2021) and attitude (Caner, 2019; Küçük, 2021; Özay Köse & Gül, 2019; Seyhan & Şimşek, 2017; Soysal, 2019), and their experiences with these environments (Can, 2019; Çalışkan & Çerkez, 2012; Çiçek & Saraç, 2017; Çil & Yanmaz, 2016; Demir & Öner Armağan, 2018). When the research are examined, it is seen that the subject area, aims, method, data collection tools, and study groups of the studies are different from each other, and each of them should be analyzed separately. Furthermore, these studies' simultaneous access, reading, interpretation, and analysis difficulties reveal the need for research based on holistic analysis in this field. Although there are review studies on out-of-school learning for this need, it is seen that their number is quite limited. When the literature was examined, a meta-analysis (Mutlu & Çelik, 2019) and two content analysis studies (Demircioğlu & Aslan, 2018; Saraç, 2017) were found. Mutlu and Çelik (2019) conducted a meta-analysis study on research conducted on out-of-school learning in science education in Turkey between 2006 and 2019. In their research, two themes were focused on: perceptions of out- of-school learning environments and the effect of out-of- school learning environments on student outcomes. In their research, Demircioğlu and Aslan (2018) investigated postgraduate studies on out-of-school learning environments in Turkey within the framework of the subject area, method, sampling, data collection tools, and data analysis methods using content analysis. In Saraç's (2017) research, the national studies on out-of-school learning environments between 2007 and 2016 were subjected to content analysis. In recent years, the number and variety of out-of-school learning activities have increased; accordingly, studies on out-of-school learning have also increased. However, no comprehensive systematic review study has been carried out recently. The research results, which include holistic analyses of out-of-school learning activities, will contribute to summarizing current studies in the relevant field and facilitate access of researchers, teachers, and other stakeholders to research data related to the subject area. In addition, the holistic data presented on the content of the studies carried out on this subject will allow researchers to see new and different studies holistically and contribute to developing different perspectives. From this point of view, this research aims to make a systematic review of the research on "out-of-school learning activities" in Turkey between 2000-2020. Therefore, all articles, master's theses, and doctoral dissertations, which have the concept of out-of-school learning in the title and keywords, and published within the specified period, were examined according to different variables. Therefore, this systematic review seeks answers to the following questions: How is the distribution of studies conducted on out-of- school learning between 2000 and 2020 by: 1. publication types? 2. years? 3. subject areas? 4. educational stages of the samples? 5. sample sizes? 6. their aims? 7. out-of-school learning environments? 8. research approaches? 9. research methods? 10. data collection tools? 11. data analysis methods? 2. METHOD 2.1 Research Design This research was carried out to determine the general trends of the studies on "out-of-school learning activities" in Turkey between 2000-2020. For this purpose, a systematic review was applied to the theses and articles published in this field within 20 years. This systematic review was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) principles (Liberati et al., 2009). The PRISMA principles with the checklist consisting of 27 items and a four-step flow chart ensure that the literature review studies are conducted transparently (Liberati et al., 2009). Search strategy and criteria for inclusion and exclusion of studies in the systematic review The inclusion criteria for the study-selection phase of the systematic review are: ⚫ The studies should be published between January 1, 2000, and December 31, 2020, and their full texts can be accessed, ⚫ The articles should be included in Google Scholar and/or TR Index databases, and their full texts should be available, ⚫ Master's theses and doctoral dissertations should be included in the Council of Higher Education National Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 512 J.Sci.Learn.2022.5(3).509-519 Thesis Center database, and their full texts should be available, ⚫ The studies should have been conducted in the field of education, ⚫ The studies should include the keywords “out-of- school learning”, “out-of-course learning”, “out-of- class learning”, “free-choice learning”, “learning outside” or “outdoor education”. When searching in Turkish by limiting the years as 2000-2020, 1140 when "out-of-school learning" is written as a keyword in the Google Scholar database, 390 when "out-of-class learning" is written, 259 when "outdoor education" is written, 101 when "out-of-course learning" is written, 14 when "learning outside" is written, and 12 articles were found when "free-choice learning" is written. In addition, when these concepts were written and searched in the CoHE National Thesis Center database, 92 theses were reached. As a result of the examination of the reached studies following the research purpose, it was decided to include 303 studies. Totally 1465 studies related to out-of-school learning were found in databases. Among these studies, 45 were excluded since their full texts are unavailable. Then, 105 remaining studies were excluded since they did not focus on education. The remaining 303 studies are involved in this research. The PRISMA process is represented in Figure 1. 2.2 Data Collection Tool A research classification form was developed to evaluate the studies about out-of-school learning. The form was used as a data collection tool in the research context. While developing the form, the related literature was investigated (Çiltaş, Güler, & Sözbilir, 2012; Selçuk, Palancı, Kandemir, & Dündar, 2014; Sözbilir, Kutu, & Yaşar, 2012), and the features that should be found in scientific research were examined (Büyüköztürk, Akgün, Karadeniz, Demirel, & Kılıç, 2016; Çepni, 2014; Metin, 2014). In the research classification form, there are nine sections: The publication type, year, purpose, subject area, out-of-school environment, research approach and method, sample group, sample size, and data collection tool. The categories in this classification form and the options given under these categories were presented in the opinion of two experts working in content analysis. The form was finalized in line with expert opinions. 2.3 Data Analysis 211 articles and 92 theses examined within the scope of the research were analyzed according to the research classification form developed by the researchers. In the research, researcher triangulation was done to ensure the validity of the data. In this framework, two researchers took part in all data collection, analysis, and interpretation processes (Merriam, 2009). During the analysis of the studies, the researchers evaluated the determined studies separately, taking into account the categories determined in the form. The findings obtained from the studies were recorded in a Microsoft Excel file and categorized. After the analysis of the studies was completed, the coding reliability was checked to ensure the reliability of the research (Miles & Huberman, 2015). The analyses made by the two researchers were compared, and the analysis data were arranged in line with the common opinion. The results obtained are expressed in graphs and presented in the findings section. 3. FINDINGS Within the scope of the research, a total of 303 studies were found. The findings obtained from the content analysis are presented under five headings: Distribution of the studies according to 1) publication types, years, and subject areas, 2) educational stages and sample sizes, 3) aims and out-of-school learning environments, 4) research Figure 1 The process of reaching the studies Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 513 J.Sci.Learn.2022.5(3).509-519 approach and methods, 5) data collection tools and analysis methods. 3.1 Distribution of the studies according to publication types, years, and subject areas Of the 303 studies examined, 211 (69.6%) were articles, 73 (24.1%) were master's theses, and 19 (6.3%) were doctoral dissertations (Figure 2). It was determined that outdoor learning studies conducted in line with the 303 studies analyzed have increased over the years (see Figure 3). The number of studies, which was one in 2002, increased to 20 in 2016 and reached the maximum number in 2020. The number of studies published in 2020 is currently 52. According to the results, 131 studies (43.2%) were carried out in science, 53 studies (17.5%) were in social studies, and 31 (10.2%) were in other subject areas-- four geography, four physical education, four physics, four special education, three biology, three life science, three Turkish, two chemistry, one information technologies, one English, one religious culture and moral knowledge, and one technology and design. Moreover, 12 studies (4.0%) were conducted in preschool education, 12 studies (4.0%) were in visual arts, six studies (2.0%) were in mathematics, and five studies (1.6%) were in history subject area. No subject area was specified in 53 studies (17.5%) (see Figure 4). 3.2 Distribution of the studies according to educational stages and sample sizes Of the 303 studies analyzed, middle school students were included in 121 studies, teachers were in 68, preservice teachers were in 43, primary school students were in 14, high school students were in 13, and preschool students were included in nine studies. In addition, parents were included in five, museum educators were in two, and academicians were included in one research (see Figure 5). In 31 studies, no educational stage was specified as they are grounded theory, meta-synthesis, and document analysis studies. Figure 2 Distribution of the studies according to the publication types 70% 24% 6% articles master's theses doctoral dissertations Figure 3 Number of articles published by years 52 49 34 33 20 18 14 16 18 7 12 8 8 6 2 2 2 1 1 0 10 20 30 40 50 60 N u m b e r o f th e s tu d ie s Years Figure 4 Distribution of the studies according to the subject areas Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 514 J.Sci.Learn.2022.5(3).509-519 The distribution of examined studies according to the sample sizes is presented in Figure 6. Accordingly, 59 studies have 1-20 participants, 73 studies have 21-40 participants, 36 studies have 41-60 participants, 28 studies have 61-80 participants, 35 studies have 81-160 participants, and 37 studies have 161 and more participants. In 35 studies, sample sizes were not specified. According to the results, most studies have 21-40 participants. 3.3 Distribution of the studies according to the aims The distribution of examined studies according to their aims is presented in Figure 7. Accordingly, 123 studies aimed to investigate participants' views regarding out-of- school learning environments, and 46 studies aimed to investigate the effects of out-of-school learning environments on the participants' academic achievement. Moreover, out-of-school learning environments' effects on the participants' attitudes and skills (critical thinking skills, scientific process skills, decision-making skills, etc.) were investigated in 38 and 19 studies, respectively. In addition, participants' self-efficacy perceptions and perceptions regarding out-of-school learning environments were investigated in 17 studies. Lastly, the effect of out-of- school learning activities on middle school students' interest in science, motivation, and anxiety was examined in seven, six, and three studies, respectively. Figure 5 Distribution of the studies according to the educational stages of the samples Figure 6 Distribution of the studies according to the sample sizes 121 68 43 31 14 13 9 5 2 1 0 20 40 60 80 100 120 140 N u m b e r o f th e s tu d ie s Educational stages 59 73 36 28 11 6 9 9 37 35 0 10 20 30 40 50 60 70 80 1-20 21-40 41-60 61-80 81-100 101-120 121-140 141-160 161 and above not specified n u m b e r o f th e s tu d ie s sample size Figure 7 Distribution of the studies according to the aims 123 46 38 19 17 7 6 3 0 50 100 150 view achievement attitude skills perception interest motivation anxiety Number of the studies a im o f th e s tu d y Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 515 J.Sci.Learn.2022.5(3).509-519 3.4 Distribution of the studies according to the out-of- school learning environments The distribution of examined studies according to the out-of-school learning environments is presented in Figure 8. When Figure 8 is examined, it is seen that the studies on out-of-school learning have been carried out in various environments. Accordingly, it is observed that the majority of the studies have been conducted in museums and science centers. Moreover, 16 studies were conducted in the planetarium, and 16 were carried out in different places such as canyon, conservatory, sports hall, and earthquake monitoring center. 3.5 Distribution of the studies according to research approach and methods The distribution of the examined studies according to the research approach and method is given in Figure 9. Accordingly, six studies (one master's thesis and five articles) are scale development studies. The quantitative research method has been used in 93 studies, the qualitative research method in 141 studies, and the mixed research method in 45 studies. Moreover, in the 18 studies, no research approach was specified. The results determined that the most preferred research methods in qualitative and quantitative studies are case studies and experimental research. However, it is seen that mixed-methods designs were generally not specified in the studies, and they are expressed only as "mixed method was used". 3.6 Distribution of the studies according to data collection tools and analysis methods The distribution of the data collection tools used in examined studies is given in Figure 10. Accordingly, a total of 68 scales, 59 questionnaires, 151 forms, 51 tests, and 38 other data collection tools were included in the studies. It is seen that primarily attitude scales (38) were used in studies where the scales were used, and mainly interview forms (122) were used in studies where forms were used. Also, mostly achievement tests (43) were preferred in the studies where the tests were used. Figure 8 Distribution of the studies according to the out-of-school learning environments Figure 9 Distribution of the studies according to research approach and methods Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 516 J.Sci.Learn.2022.5(3).509-519 The distribution of the analysis methods of the examined studies is given in Figure 11. Accordingly, content analysis was conducted in 106 studies, and descriptive analysis was conducted in 71. Moreover, t-tests were used in 68 studies, descriptive statistics were used in 67 studies, nonparametric tests were used in 38 studies, and ANOVA was used in 34 studies. 4. DISCUSSION In the light of the findings obtained from the research, it is seen that the articles published on out-of-school learning environments outnumbered doctoral dissertations and master's theses. This finding parallels the findings obtained in Saraç's (2017) study. In his research, Saraç (2017) examined 133 studies conducted in the national field and published between 2007 and 2016 on out-of-school learning environments and concluded that 76 of them were articles. In addition, Saraç (2017) and Demircioğlu and Aslan (2018) concluded that the number of master's theses published on out-of-school learning in Turkey is much more than the doctoral dissertations. The findings of the present research are also in this direction. Of the 303 studies examined, 211 were articles, 73 were master's theses, and only 19 were doctoral dissertations. According to the findings, the highest number of publications between 2000 and 2020 on out-of-school learning were made in 2020, and the studies have increased over the years. Moreover, substantial increases in 2012, 2017, and 2019 were detected. The leap in 2012 was also detected in Saraç's (2017) studies and Demircioğlu and Aslan's (2018) studies. In the increase in the number of studies in 2019, it is thought that the addition of the professional knowledge elective course called "out-of- school learning environments" to the education faculties in 2018 by the Council of Higher Education also played a role. In addition to this course, the out-of-school learning environments course was added as a compulsory course in some departments such as science teaching and primary school mathematics teaching in 2018 (Council of Higher Education [CoHE], 2018a, 2018b). This situation may have increased the interest and focus of researchers and educators on out-of-school learning after 2018. The results revealed that most of the reviewed studies were conducted in the science subject area. Social studies course comes after science. On the other hand, the less common subject areas are found in visual arts, mathematics, and history. In these curricula, out-of-school learning is not emphasized. Similar to the present research results, according to Saraç’s (2017) and Demircioğlu and Aslan's (2018) studies, the two most common subject areas are science education and social studies, respectively. The research of Ertuğrul and Karamustafaoğlu (2020) also supports these results. Figure 10 Distribution of the studies according to data collection tools Figure 11 Distribution of the studies according to the analysis methods 106 71 68 67 38 34 12 8 6 0 50 100 150 content analysis descriptive analysis t-tests descriptive statistics nonparametric tests ANOVA other ANCOVA factor analysis Number of the studies A n a ly s is m e th o d s Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 517 J.Sci.Learn.2022.5(3).509-519 Furthermore, the researchers investigated the views of classroom teachers towards out-of-school learning environments. Their findings revealed that classroom teachers thought out-of-school learning environments should be visited within science and social studies lessons. It is thought that the Ministry of National Education stated in the 2013 and 2018 science curriculums that out-of- school learning environments can also be used for students to learn knowledge meaningfully and permanently is thought to be effective in obtaining these findings (MoNE, 2013, 2018b). The results obtained from the analysis indicated that middle school students and teachers were included in most of the studies. On the other hand, preschool students, parents, museum educators, and academicians were included in only a few studies. Parallel to this finding, Demircioğlu and Aslan (2018) also found that the studies are mostly carried out with middle school students and teachers. It is thought that the recent changes in both the science curriculum (Ministry of National Education [MoNE], 2013, 2018b) and teacher education program (CoHE, 2018a, 2018b), whereby out-of-school learning environments have been included in teaching plans have brought along the need to work more frequently with that sample group. The results revealed that the number of sample groups studied is not much high. Accordingly, most studies have 1-20 and 21-40 participants. Similarly, in the study of Demircioğlu and Aslan (2018), most of the theses have 10- 50 participants. This preference is assumed to be linked to the recommendation that studies should be conducted with a smaller number of sample groups in order to make trips to out-of-school learning contexts more effective. However, this situation might also be due to the shortage of time and official procedures. Furthermore, it is known that working with large sample groups in out-of-school learning environments can cause various problems such as a decrease in efficiency, ensuring safety, and ethical problems (Karbeyaz & Kurt, 2020; Ocak & Korkmaz, 2018). Therefore, according to the findings obtained in the present research, the most frequently used data collection tool is the interview form. Additionally, the findings regarding the distribution of the studies according to the aims indicated that of the 303 studies, 123 studies aimed to investigate participants’ views regarding out-of-school learning environments. Therefore, it can be said that the most commonly used sample size matches the most frequently preferred data collection tools and aims of the studies. It was revealed that most studies were conducted in museums and science centers. This finding parallels the other studies (Demircioğlu & Aslan, 2018; Saraç, 2017). Since the studies are mainly conducted in science and social studies subject areas, selecting these environments makes sense. However, when the studies were examined in more detail, it was seen that the studies on science education were mostly carried out in science centers, and the studies on social studies were mainly carried out in museums. Interview forms, questionnaires, and achievement tests are the most used data collection tools in the studies. In parallel with these findings, Saraç’s (2017) research found that questionnaires and interview forms are the most used data collection tools in the investigated studies. In the research of Demircioğlu and Aslan (2018), two of the most commonly preferred data collection tools were interviews and achievement tests. Concordant with these results, in the present research, most of the studies aimed to investigate participants' views regarding out-of-school learning environments and to investigate the effects of out- of-school learning environments on the participants' academic achievement. On the other hand, the effect of out-of-school learning activities on students' interest in science, motivation, and anxiety was examined in only a few studies. As a result, the most commonly used data collection tools match the aims of the studies. The results indicated that the qualitative research method was used in most reviewed studies while the mixed method is relatively less preferred. This result is parallel with the literature (Saraç, 2017). Çiltaş et al. (2012) also stated that mixed studies are scarce in our country's field of education. The detailed analyses determined that the most preferred research methods in qualitative and quantitative studies are case studies and experimental research. Concordantly, Demircioğlu and Aslan (2018) had precisely the same results in their research. In parallel with these findings, Saraç (2017) also found that the reviewed qualitative studies were mostly carried out with case studies. Consistent with the most used research methods; content analysis, descriptive analysis, t-tests, and descriptive statistics are the studies' most used four analysis methods. RECOMMENDATION In the light of the findings, some suggestions are provided for researchers who may perform studies on similar subjects in the future: ⚫ Remarkably, there are very few doctoral dissertations conducted in out-of-school learning environments. Therefore, the number of doctoral dissertations should be increased. ⚫ The results indicated a high number of studies performed on science and social studies education. Therefore, it is anticipated that new studies on out-of- school learning environments in other branches such as history, mathematics, and preschool education will contribute to the literature. ⚫ According to the results, middle school students and teachers were included in most studies; however, preschool students and parents were included in only a few studies. Since people of all ages visit out-of- Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 518 J.Sci.Learn.2022.5(3).509-519 school learning environments, choosing diverse sample groups for the studies would benefit the researchers. ⚫ The number of samples in the studies may be increased to reach more data and more accurate results. ⚫ Out-of-school learning environments should not be limited to science centers and museums. Visiting different learning environments would increase students’ motivation and interest. ⚫ It is considered that using the mixed method and the qualitative and quantitative methods will be effective in the studies carried out. ⚫ Lastly, it is suggested to conduct studies that compare out-of-school learning in Turkey and other countries. REFERENCES Akça, Z., Balkan Kıyıcı, F., & Yıldız, A. (2017). The effect of use of conservatories in the teaching of voice subject on the academic achievements and thinking skills of students. Journal of Current Researches on Educational Studies, 7(2), 171-184. Akman, B., Özen Altınkaynak, Ş., Ertürk Kara, H. G., & Can Gül, Ş. (2015). The views of preschool teachers on museum education. Journal of Uludag University of Faculty of Education, 28(1), 97-115. Ayotte-Beaudet, J. P., Potvin, P., Lapierre, H. G., & Glackin, M. (2017). Teaching and Learning Science Outdoors in Schools’ Immediate Surroundings at K-12 Levels: A Meta-Synthesis. EURASIA Journal of Mathematics Science and Technology Education, 13(8), 5343-5363. http://dx.doi.org/10.12973/eurasia.2017.00833a Avcı, G., & Gümüş, N. (2019). Views of students on outdooroutdoor education. Turkish Studies-Educational Sciences, 14(3), 351-377. Ay, Y., Anagün, Ş. S., & Demir, Z. M. (2015). Preservice primary school teachers' opinions about out-of-school learning in science teaching. Turkish Studies- International Periodical for the Languages, Literature and History of Turkish or Turkic, 10(15), 103-118. Aydemir, İ., & Toker Gökçe, A. (2016). Perspectives of school administrators regarding informal learning/education settings. In Proceedings of 3rd International Eurasian Educational Research Congress (pp. 630-638). Bakioğlu, B., & Karamustafaoğlu, O. (2020). Pupil views on the use of out-of-school learning environments in teaching process. Journal of Research in Informal Environments, 5(1), 80-94. Batman, D. (2020). Investigation of physics teachers’ views about the out-of-school learning environments. Journal of Research in Informal Environments, 5(1), 59-79. Bell, P., Lewenstein, B., Shouse, A.W., & Feder, M.A. (Eds.). (2009). Learning science in informal environments. Washington, DC: National Research Council. Bilir, M. (2007). Non-formal education implementations in Turkey: Issues and latest challenges. International Journal of Lifelong Education, 26(6), 621-633. Binbaşıoğlu, C., (2000). Extracurricular activities at school. MoNE Teacher’s Book Series, National Education Press, İstanbul. Bolat, A., Karamustafaoğlu, S., & Karamustafaoğlu, O. (2020). The effect of outdooroutdoor school learning environment on student achievement in 5th grade 'World of Living' unit: Example of biodiversity museum. Karaelmas Journal of Educational Sciences, 8, 42- 54. Bostan Sarıoğlan, A., & Küçüközer, H. (2017). Investigation of preservice science teachers' opinions regarded to outdooroutdoor school learning environments. Journal of Research in Informal Environments, 2(1), 1-15. Bozdoğan, A. E. (2007). Role and importance of science and technology in education. (Unpublished doctoral dissertation). Gazi University, Ankara. Bozdoğan, A. E. (2008). Planning and evaluation of field trips to informal learning envıronments: Case of the ‘Energy Park’. Journal of Theory and Practice in Education, 4(2), 282-290. Bozdoğan, A. E. (2017). Preservice teachers' views about "Informal Learning Environments in Science Education" course. International Journal of Turkish Education Sciences, 5(8), 1-17. Bozdoğan, A. E., & Kavcı, A. (2016). The effects of out of class teaching activities to secondary school student's academic achievement in science course. Gazi Journal of Education Sciences, 2(1), 13-30. Braund, M., & Reiss, M. (2006). Towards a more authentic science curriculum: The contribution of out-of-school learning. International Journal of Science Education, 28, 1373-1388. http://dx.doi.org/10.1080/09500690500498419 Bülbül, M. (2018). The effect of non-school learning environments on the academic achievements of students: The example of the hydroelectric power plant trip. (Unpublished master’s thesis). Giresun University, Giresun. Büyükkaynak, E., Ok, Z., & Aslan, O. (2016). Science teachers’ views on out-of-school learning environments in science education. Kafkas University Journal of the Institute of Social Sciences, 1, 43-60. Büyüköztürk, Ş., Akgün, Ö. E., Karadeniz, Ş., Demirel, F., & Kılıç, E. (2016). Scientific research methods. Ankara: Pegem Academy. Can, N. S. (2019). The investigation of out-of-school activities of learning environments on recycling and environmental effects in primary school students in terms of different variables. (Unpublished master’s thesis). Erzincan Binali Yıldırım University, Erzincan. Caner, Ö. (2019). Teacher candidates’ attitudes towards the sustainable environmental education in out-of-school learning environments. (Unpublished master’s thesis). Akdeniz University, Antalya. Council of Higher Education [CoHE]. (2018a). Science education undergraduate program. Retrieved from https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_ dairesi/Yeni-Ogretmen-Yetistirme-Lisans- Programlari/Fen_Bilgisi_Ogretmenligi_Lisans_Programi.pdf CoHE. (2018b). Elementary mathematics teaching undergraduate program. Retrieved from https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_ dairesi/Yeni-OgretmenYetistirme-Lisans- Programlari/Ilkogretim_Matematik_Lisans_Programi.pdf Cross, J. (2007). Informal learning: Rediscovering the natural pathways that inspire Innovation and Performance. San Francisco: Pfeiffer. Çalışkan, H., & Çerkez, S. (2012). An evaluation of education with museum practices in social studies classes based on student views. International Online Journal of Educational Sciences, 4(1), 162-173. Çepni, S. (2014). Introduction to research and project work. Ankara: Pegem A Publishing. Çiçek, Ö., & Saraç, E. (2017). Science teachers' opinions about experience in out of school learning environments. Ahi Evran University Journal of Kırşehir Education Faculty, 18(3), 504-522. Çil, E., & Yanmaz, D. (2016). Primary education students’ museum visit experiences. In Proceedings of 15th International Primary Teacher Education Symposium (pp. 860-869), Muğla, Turkey. Çiltaş, A., Güler, G., & Sözbilir, M. (2012). Mathematics education research in Turkey: A content analysis study. Educational Sciences: Theory & Practice, 12(1), 574-578. Demir, N., & Öner Armağan, F. (2018). Science teachers’ views about informal learning environments: Planetarium. Journal of Social and Humanities Sciences Research, 5(30), 4241-4248. Demircioğlu, G., & Aslan, A. (2018). A review on Turkish graduate studies performed on out-of-school learning environments. Karadeniz Technical University Institute of Social Sciences Journal of Social Sciences, 8(16), 379-402. Dori, Y. J., & Tal, R. T. (2000). Formal and informal collaborative projects: Engaging in industry with environmental awareness. Science Education, 84(1), 95-113. http://dx.doi.org/10.12973/eurasia.2017.00833a http://dx.doi.org/10.1080/09500690500498419 https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Yeni-Ogretmen-Yetistirme-Lisans-Programlari/Fen_Bilgisi_Ogretmenligi_Lisans_Programi.pdf https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Yeni-Ogretmen-Yetistirme-Lisans-Programlari/Fen_Bilgisi_Ogretmenligi_Lisans_Programi.pdf https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Yeni-Ogretmen-Yetistirme-Lisans-Programlari/Fen_Bilgisi_Ogretmenligi_Lisans_Programi.pdf https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Yeni-OgretmenYetistirme-Lisans-Programlari/Ilkogretim_Matematik_Lisans_Programi.pdf https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Yeni-OgretmenYetistirme-Lisans-Programlari/Ilkogretim_Matematik_Lisans_Programi.pdf https://www.yok.gov.tr/Documents/Kurumsal/egitim_ogretim_dairesi/Yeni-OgretmenYetistirme-Lisans-Programlari/Ilkogretim_Matematik_Lisans_Programi.pdf Journal of Science Learning Article DOI: 10.17509/jsl.v5i3.45460 519 J.Sci.Learn.2022.5(3).509-519 Erten, Z., & Taşçi, G. (2016). Developing activities of out of the school learning environments for science classes, and analysing their effects on students’ scientific process skills. Erzincan University Journal of Education Faculty, 18(2), 638-657. Ertuğrul, A., & Karamustafaoğlu, O. (2020). Views of classroom teachers about out-of-school learning environments: Kayseri Science Center. Social Sciences Research Journal, 9(2), 107-116. Eshach, H. (2007). Bridging in-school and out-of-school learning: formal, non-formal, and informal education. Journal of Science Education and Technology, 16, 171-190. Fidan, N. (2012). Learning and teaching at school (3rd Ed.). Pegem Academy, Ankara. Güngör, C., & Göloğlu Demir, C. (2022). Analysis of preschool teachers' views on out of school learning activities. Afyon Kocatepe University Journal of Social Sciences, 24(1), 15-30. Güven, İ. (2010). Türk eğitim tarihi [Turkish education history]. Naturel Yayıncılık. Higgins, P., Loynes, C., & Crowther, N. (Eds.). (1997). A guide for outdoor educators in Scotland. SNH: Perth. Karademir, E.(2013). Determination of objectives realization at outdooroutdoor science education activities of teachers and preservice teachers by the theory of planned behavior within the scope of science and technology lesson. (Unpublished doctoral dissertation). Hacettepe University, Ankara. Karakaş Özür, N., & Şahin, S. (2017). The effect of out of classroom activities on student success in the social studies course. Ahi Evran University Journal of Kırşehir Education Faculty, 18(3), 324-347. Karamustafaoğlu, S., Ayvalı, L., & Ocak, Y. (2018). Teachers' opinions on informal environments in preschool education. Journal of Research in Informal Environments, 3(2), 38-65. Karbeyaz, A., & Kurt, M. (2020). The views of classroom teachers for use of out-of-school learning environments in life studies course. International Journal of New Approaches in Social Studies, 4(1), 79-93. Kubat, U. (2018). Opinions of preservice science teachers about outdooroutdoor education. Mehmet Akif Ersoy University Journal of Education Faculty, 48, 111-135. Küçük, A. (2021). Investigation of the change towards scientific attitudes of students with out-of-school learning experience. OPUS International Journal of Society Researches, 18(44), 7552-7580. Küçük, A., & Yıldırım, N. (2021). The effect of out-of-school learning environments on the academic achievement of the human and environment unit. Fen Bilimleri Öğretimi Dergisi, 9(2), 205-264. Laçin Şimşek, C. (2011). Out-of-school learning environments and science education. In Laçin Şimşek (Ed.), Out-of-school learning environments in science teaching (pp. 1-23). Pegem Academy, Ankara. Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P., ... & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of clinical epidemiology, 62(10), e1-e34. Lockhart, A. S. (2016). Non-formal and informal programs and activities that promote the acquisition of knowledge and skills in areas of Global Citizenship Education (GCED) and Education for Sustainable Development (ESD). UNESDOC: UNESCO Digital Library. Retrieved February 21, 2021 from https://unesdoc.unesco.org/ark:/48223/pf0000245625 Melber, L. H., & Abraham, L. M. (1999). Beyond the classroom: Linking with informal education. Science Activities, 36, 3-4. Merriam, S. B. (2009). Qualitative research: A guide to design and implementation (2nd Ed.). San Francisco, CA: Jossey-Bass. Metin, M. (2014). Nicel Veri Toplama Araçları [Quantitative Data Collection Tools]. In M Metin (Ed.), Kuramdan Uygulamaya Eğitimde Bilimsel Araştırma Yöntemleri (pp.161-214), Ankara: Pegem A Yayıncılık. Metin, M., & Özcan, A. (2015). Basic concepts of education. In Metin, M. & Aytaç T. (Eds.), Introduction to educational science. Pegem-A Publishing, Ankara. Miles, M., & Huberman, A. M. (2015). Qualitative data analysis: A resource book challenging new methods. Jakarta (ID): UI Press. Ministry of National Education [MoNE]. (2005). Primary schools science and technology education program for grades 4 and 5. Ankara. MoNE. (2013). Elementary science course (4, 5, 6, 7 and 8th grades) curriculum. Ankara: MoNE Publishing. MoNE. (2018a). 2023 education vision program. Ankara: MoNE Publishing. MoNE. (2018b). Science curriculum (primary and middle school 3, 4, 5, 6, 7 and 8th grades). Ankara. Mutlu, G., & Çelik, M. (2019). Research conducted about out-of-school learning environments in science education in Turkey: A meta- synthesis study. In Proceedings of 1st International Educational Research Conference (pp. 159-169), Izmir, Turkey. Ocak, İ., & Korkmaz, Ç. (2018). An examination of the views of science and preschool teachers on non-formal learning environments. International Journal of Field Education, 4(1), 18-38. Öner, G., & Öztürk, M. (2019). Science centres as outdooroutdoor teaching environments: Experience of prospective social studies teachers. Eskisehir Osmangazi University Journal of Social Sciences, 20, 1109-1135. Özay Köse, E., & Gül, Ş. (2019). Investigation of high school students' attitudes about botanical gardens according to different variables. International Journal of Humanities and Education, 5(12), 1019-1036. Ramey-Gassert, L. (1997). Learning science beyond the classroom. The Elementary School Journal, 97(4), 433-450. Saraç, H. (2017). Researches related to outdooroutdoor learning environments in Turkey: Content analysis study. Journal of Education Theory and Practical Research, 3(2), 60-81. Selçuk, Z., Palancı, M., Kandemir, M. & Dündar, H. (2014). Tendencies of the researches published in Education and Science Journal: Content analysis. Education and Science, 39(173), 430-453. Seyhan, A., & Şimşek, U. (2017). The effects of using cemeteries as an educational stage on students’ academic success and attitudes in a high school history course. Turkish History Education Journal, 6(2), 343-362. Soysal, E. (2019). The effects of out of school learning environment on attitude, interest and motivation toward science course of 7th grade students. (Unpublished master’s thesis). Gazi University, Ankara. Sözbilir, M., Kutu, H., & Yaşar, M. D. (2012). Science education research in Turkey: A content analysis of selected features of papers published. In J. Dillon & D. Jorde (Eds). The World of Science Education: Handbook of Research in Europe (pp.341-374). Rotterdam: Sense Publishers. Strauss, L., & Terenzini, P. (2007). The effects of students' in- and out- of-class experiences on their analytical and group skills: A study of engineering education. Research in Higher Education, 48(8), 967-992. Şen, A. İ. (2019). What is an out-of-school learning environment? In A. İ. Şen (Ed.) Out-of-school learning environments (pp.1-20). Ankara: Pegem Academy Publishing. Şimşek, A., & Kaymakçı, S. (2015). The purpose of out-of-school social studies education. In Şimşek & Kaymakçı (Eds), Out-of-school social studies education (pp. 1-11). Ankara: Pegem A. Tal, T., & Morag, O. (2009). Reflective practice as a means for preparing to teach outdoors in an ecological garden. Journal of Science Teacher Education, 20, 245-262. Türkmen, H. (2015). Primary teachers' point of view about science teaching in outdooroutdoor learning environments. Journal of European Education, 5(2), 47-55. Türkmen, H. (2018). Perspectives of secondary school teachers about outdooroutdoor teaching. Journal of Ege Social Science, 1(1), 12-26. Uludag, G. (2021). Views of preschool teachers on using out-of-school learning environments in preschool education. International Online Journal of Education and Teaching, 8(2), 1225-1249. UNESCO, UNDP, UNFPA, UNHCR, UNICEF, UN WOMEN, … ILO. (2015). Education 2030: Incheon declaration and framework for action for the implementation of sustainable development goal 4: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. Retrieved February 21, 2021 from http://unesdoc.unesco.org/images/0024/002456/245656e.pdf 4 https://unesdoc.unesco.org/ark:/48223/pf0000245625 http://unesdoc.unesco.org/images/0024/002456/245656e.pdf%204